Tới nội dung

Lĩnh vực

Kỹ thuật triển khai Mới nhất

Chạy mô hình nhanh hơn và rẻ hơn: tăng tốc suy luận, bộ nhớ và chi phí, phục vụ mô hình và lựa chọn hạ tầng.

62 tin chọn34 trong 30 ngày quaĐã thu thập 472

Cập nhật 13 tháng 11 08:32

Kỹ thuật triển khai · Tuyển chọn · trang 2

9 tháng 11Thứ HaiMục 21–40 trong 62
  1. Google Developers Blog62

    Google makes Pathways generally available on Google Cloud for large-model training

    Google announced the General Availability of Pathways, its distributed training system for large models including Gemini, on Google Cloud Platform. Pathways replaces lockstep SPMD execution with a single Python client that declares the whole computation as a dataflow graph, while a distributed runtime places blocks on virtual devices, moves and reshards tensors, and orders work across the cluster. Google says the design lets a failed host be remapped onto healthy chips instead of restarting the job, and that a shared Pathways service can serve several clients from one cluster.

    Vì sao đáng chú ý: Google explains how Pathways' single-controller model differs from SPMD and how it handles cross-group tensor transfer, scheduling and host failure recovery.

6 tháng 11Thứ Sáu
  1. vLLM Blog77

    REINFORCE Zero: Kernel Alignment Enables Stable Async RL at Policy Lag 64, Extends to DeepSeek-V4.1 and Kimi K3.1

    NVIDIA Research and Inferact's REINFORCE Zero paper shows that aligning rollout and trainer kernels lets unclipped token-level importance sampling correct the remaining weight lag, training stably for 20,000 updates at policy lag 64 and reaching 2.40x the training-step throughput of synchronous training with aligned kernels. With standard kernels, the median rollout-to-learner KL rises 83.33x across the run and online AIME24 pass@1 falls from 34.27% to 13.96%, while a sequence gate stabilizes training but rejects 24.48% of sequences by the last 1,000 updates. The zero-mismatch setup now covers DeepSeek-V4.1-Flash, Kimi K3.1 on release day, and DeepSeek-V4-Flash and Kimi K3.1 on AMD MI355X, with code in Inferact/zero-mismatch-rl and NVIDIA-NeMo/labs-molt.

    Vì sao đáng chú ý: NVIDIA Research and Inferact split the async RL rollout-learner gap into weight lag and kernel mismatch, showing that aligning kernels lets unclipped importance sampling hold stable training at policy lag 64.

  2. Google Developers Blog60

    Google study finds stale TODO bug references distract coding agents in 13% of sessions

    ) references. Agents acted on the bug ID in 6% of encounters, and 83% of those actions were unprompted by the developer; 41% of unprompted detours chased already closed tickets while 57% hit still-open bugs and expanded scope. Controlling for developer, codebase complexity, task type, model tier and other factors, unprompted bug detours added 70% more steps, 77% more tokens and 105% more wait time per session.

    Vì sao đáng chú ý: Google's large-scale trace analysis shows stale TODO bug references pull coding agents into unprompted detours, and its cleanup agents removed thousands of obsolete references with over 95% reviewer acceptance.

5 tháng 11Thứ Năm
  1. Mistral AI69

    Mistral expands Studio with managed agents, workflows and apps alongside Mistral Large 4 (Le Chonk)

    Mistral expanded its enterprise AI platform Studio with managed agents and apps, plus new workflow, enterprise context, agent observability and evaluation capabilities, running on the Mistral Large 4 (Le Chonk) model. Developers can build multi-step agents with skills, sandboxes, subagents and hooks, and deploy to Studio with one command through the new Mistral MCP server. Mistral says customers including ABANCA, BNP Paribas, Capgemini and TCS already use Studio in production, with deployments available in the cloud, VPC or on-premises.

    Vì sao đáng chú ý: Mistral expands Studio with managed agents, workflows and apps, showing how enterprises can keep deployment and data control in their own environments.

4 tháng 11Thứ Tư
  1. vLLM Blog73

    Buoyant releases Kinference, an open source inference control plane for vLLM and Kubernetes

    Buoyant announced Kinference, an Apache v2 open source control plane that routes OpenAI- and Anthropic-compatible inference traffic to vLLM workers on Kubernetes, adding token rate limiting, per-tenant usage and cost tracking, KV-cache-aware load balancing, and YAML inference policies. In the project's public benchmarks, Kinference cut median time-to-first-token by 95% and median end-to-end latency by 45% versus standard Kubernetes service load balancing on one workload, and served 7.4x the goodput on shared-prefix and about 3.5x on multiturn in a capacity test. Policies can route simpler prompts to cheaper models, burst overflow to providers such as Bedrock and Databricks, or keep PII-tagged requests inside the cluster, and per-tenant metrics cover token rates, TTFT, ITL and cached versus uncached prompt tokens.

    Vì sao đáng chú ý: Buoyant's Kinference adds KV-cache-aware load balancing and YAML inference policies on top of vLLM, with benchmarks claiming large TTFT and goodput gains over standard Kubernetes service routing.

3 tháng 11Thứ Ba
  1. OpenAI Alignment Blog60

    OpenAI details training activation classifiers for safety and alignment

    OpenAI describes training activation classifiers, or probes, that read a model's internal residual stream activations to flag misaligned or policy-violating assistant completions, and says it has deployed them on internal and external traffic of its most capable models starting from GPT-5.6. The probes use an Attention probe architecture with MLP plus QV-attention and an exponential moving average, are trained on up to 1M messages labeled by reasoning-model judges, and run in streaming fashion on the inference engine so a completion can be blocked mid-turn. OpenAI reports excellent probe transfer across many post-training stages but more limited transfer across pre-training checkpoints, and says probes were overall more robust to jailbreaks than content-based classifiers in its red-teaming.

    Vì sao đáng chú ý: OpenAI details how activation probes on internal model activations serve as a streaming safety layer, including their limits on transfer across checkpoints and retraining needs.

2 tháng 11Thứ Hai
  1. Anthropic Research70

    Anthropic releases FlashProve, an open-source zkSNARK system for provable LLM inference

    Anthropic released FlashProve, an open-source prototype that uses zkSNARKs to prove a response came from a committed model without revealing its weights. On Llama 4 Maverick (400B-parameter MoE, 4-bit weights), a single H100 generates proofs at about 110 tokens/second with ~19 MB proofs per 1,024-token request, roughly 70-95x the GPU time vLLM needs for the same work. The release includes executable prover and verifier code, a C++ fast verifier, a Lean verifier with machine-checked completeness, soundness and zero-knowledge statements, and a technical report, all under Apache 2.0.

    Vì sao đáng chú ý: Anthropic's FlashProve shows zkSNARK-based proofs of inference are becoming practical for a 400B-parameter MoE model, with open code and Lean-checked security claims.

27 tháng 10Thứ Ba
  1. Replit Blog75

    Replit Cloud Agents now let non-engineers contribute to company codebases

    Replit launched Cloud Agents for existing repositories, letting anyone on a team describe a change in plain language and have the agent open a tested pull request in GitHub or GitLab, available to all Pro and Enterprise customers. The agent runs each change in isolated cloud sandboxes, clicks through the running app, and attaches screenshots and video to the PR, while shared environments give every contributor a preconfigured setup that updates from the main branch. Replit also introduced early-access self-improving workspace memory, which turns review comments into team skills that engineers approve, and a Slack integration that lets users mention @Replit in a thread to start a change.

    Vì sao đáng chú ý: Replit 把 Cloud Agents 接入企业既有代码库，让非工程人员以带测试视频的 PR 形式提交改动，展示了 AI 编码工具向团队协作流程延伸的一种路径。

  2. Preferred Networks News60

    PFN releases PLaMo 3.1 Prime with Web search and Deep Research

    Preferred Networks began offering PLaMo 3.1 Prime on October 27, 2026, adding Web search and a Deep Research feature that autonomously plans, collects, analyzes and writes source-cited reports over several minutes to tens of minutes. Deep Research also runs on-premises and can connect to internal document search systems, while the company reports an average ~15% score improvement across eight hallucination-related benchmarks including AA Omniscience versus PLaMo 3.0 Prime. PFN also cites 1.7x faster processing via speculative decoding and a 90% reduction in output tokens in non-reasoning mode, and launched a free PLaMo API plan plus a PLaMo Chat campaign offering Deep Research twice daily until November 30, 2026.

    Vì sao đáng chú ý: PFN's PLaMo 3.1 Prime adds autonomous Deep Research that runs on-premises, giving Japanese enterprises a locally deployable option for source-cited investigation work.

22 tháng 10Thứ Năm
  1. Modal Blog71

    Modal releases Spindle, an open-source Tinker-compatible training server

    Modal released Spindle, an open-source, Tinker-compatible training server that runs in a customer's Modal workspace, letting teams customize the stack and pay for GPU time instead of tokens. Spindle supports full-parameter training and multi-LoRA across open models, and Modal reports that sharing one trainer across 8 clients cut cost per 1M output tokens by up to 80% versus Tinker on a DAPO-math GRPO job. Modal says Spindle is still in beta and positions it between its Dojo training library and raw cluster primitives.

    Vì sao đáng chú ý: Modal's open-source Spindle server offers teams a Tinker-compatible alternative that trades per-token pricing for GPU time, with multi-LoRA sharing reported to cut costs by up to 80%.

20 tháng 10Thứ Ba
  1. Mistral AI82

    Mistral releases Voxtral 3 speech models and voice agent SDK

    Mistral released Voxtral 3, a family of speech-to-text and text-to-speech models plus a voice agent SDK built on LiveKit that streams listening, reasoning, tool calling and speaking in one pipeline. Voxtral Small Transcribe 3 covers 50+ languages with diarization and context biasing, Voxtral Mini TTS 3 supports 35+ languages and 50+ dialects, and the stack is available through Mistral's EU-hosted API or self-deployment. Pricing starts at $0.0035/min for batch transcription and $13 per 1M characters for TTS.

    Vì sao đáng chú ý: Mistral's Voxtral 3 pairs multilingual speech models with a LiveKit-based voice agent SDK, giving teams a sovereign-deployable option for real-time voice agents.

  2. Google Developers Blog60

    Google 将 TorchTitan 接入 TorchTPU，并发布 TPU 模型优化手册

    Google released a TPU fork of Meta's TorchTitan that runs PyTorch training on Cloud TPUs through TorchTPU's native torch.device("tpu") backend, with TPU-native kernels and a step-by-step optimization playbook. On TPU v7x (Ironwood), Llama 3 405B holds about 57% MFU from 128 to 1,024 chips, Qwen3 235B reaches 32.42% MFU on one 1,024-chip slice, and FLUX.1 reaches 19.4% MFU on 1,024 chips at 93% weak-scaling efficiency. The fork is at https://github.com/google-pytorch/torchtitan/, and a Transformers modeling backend plus an agent onboarding skill extend the same trainer to other checkpoints.

    Vì sao đáng chú ý: Google 与 Meta 合作把 TorchTitan 移植到 TPU，并给出从 FLUX.1 到 Qwen3 235B 的逐级调优路径与 MFU 数字，可作为 PyTorch 团队评估 TPU 训练可行性的参考。

15 tháng 10Thứ Năm
  1. Anthropic · Claude.dev62

    How the Claude Code team does code review

    The Claude Code team describes its layered code review flow, in which most reading is done by machines and human attention is reserved for risk-based approval. From June to September, merged pull requests to the Claude Code codebase more than doubled while the share that had to be reverted fell by more than 40 percent. The process stacks verification before the PR, required CI, Claude review, security review, risk-tiered human approval and a merge queue with flags and fast rollback, and the team turned past incidents into a daily eval set for the reviewer.

    Vì sao đáng chú ý: Anthropic's Claude Code team details how it moved most code reading to automated layers while merged PRs doubled and reverts fell over 40 percent, offering a transferable review model for agent-heavy teams.

  2. Modal Blog78

    Meta's Muse Spark Open weights released and available on Modal Auto Endpoints

    Meta released the weights for Muse Spark Open, and Modal now serves the model through its Auto Endpoints with per-token billing. The model targets agentic coding and long-horizon tasks, with native multimodal understanding, a 1M-token context window, multi-step planning, sequential tool use and failure recovery. Meta reports 76.4 on Terminal-Bench 2.1 with the Terminus 2 agent and 60 on DeepSWE 1.1 using its preferred harness at xhigh reasoning effort.

    Vì sao đáng chú ý: Meta's open-weight Muse Spark Open is now served on Modal, giving teams a per-token way to try an agentic coding model with a 1M-token context window.

14 tháng 10Thứ Tư
  1. Modal Blog71

    Modal crosses $1B annualized run rate and raises $750M Series D at $15.75B valuation

    Modal crossed $1B in annualized run rate and raised a $750M Series D led by Accel at a $15.75B post-money valuation, with all existing investors participating. The company says usage has tripled since its Series C five months ago and grown more than 15x year-over-year, and that its Sandboxes product alone is at a $300M run rate with close to 3 billion sandboxes spun up. Modal also describes rebuilding its own file system, container runtime, scheduler and networking on a capacity pool spanning 20+ clouds, and points to recent launches in multi-node training, Modal servers and generally available VMs for Sandboxes.

    Vì sao đáng chú ý: Modal's post explains why AI workloads pushed it to rebuild its own file system, container runtime and scheduler across 20+ clouds, a concrete view of how the AI infrastructure stack is diverging from the cloud stack.

13 tháng 10Thứ Ba
  1. Amazon Science75

    Amazon Web Services researchers propose priming to convert pretrained Transformers into hybrid state space models without full retraining

    Amazon Web Services researchers introduce priming, a method that reuses pretrained Transformer weights to cut the pretraining cost of hybrid state space architectures by more than 99%, using under 0.5% of the source model's pretraining token budget. In controlled comparisons, Gated KalmaNet outperformed Gated DeltaNet and Mamba-2, and a 32B GKA-based hybrid matched its source Transformer on reasoning benchmarks while decoding up to twice as quickly at 128,000 tokens of context. The team released 8B and 32B primed hybrid models, training code, Triton kernels, and a vLLM serving plugin under the Apache 2.0 License.

    Vì sao đáng chú ý: Amazon Web Services researchers show that reusing pretrained Transformer weights can cut hybrid architecture pretraining cost by over 99%, making controlled comparisons of state space layers practical.

8 tháng 10Thứ Năm
  1. Anthropic · Claude.dev62

    Anthropic walks through building scheduled agent automations with Claude Managed Agents

    Anthropic published a reference implementation for Claude Managed Agents (beta) that reads Slack channels and GitHub pull requests on a cron schedule, tracks changes since the last run via per-source bookmarks, and posts a brief to Slack. The walkthrough covers vault-held credentials, read-only access and a per-run spending cap, plus rules for reporting a failed source as unreadable rather than a quiet day and confirming a Slack post before updating bookmarks and the ledger. A Claude Code command with the claude-api skill configures the agent from the article's guidance.

    Vì sao đáng chú ý: Anthropic shares a reference implementation for scheduled Claude Managed Agents that read Slack and GitHub, with concrete rules for bookmarks, failed reads and post confirmation.

  2. Goodfire Research71

    Goodfire trains and deploys probe-based cyber misuse monitors on Kimi K3 and GLM 5.3

    3 and deployed them on a production inference stack, where a cheap probe filters turns and only suspicious exchanges reach an LLM judge. 5% benign-session interruption rate, costs $65 per million turns without judge reasoning and $185 with it, and adds no measurable throughput loss inside SGLang.

    Vì sao đáng chú ý: Goodfire shows a probe-plus-judge cascade can monitor agentic cyber misuse at roughly 50x lower cost with no measurable throughput loss, a concrete recipe for serving open models safely.

6 tháng 10Thứ Ba
  1. Anthropic · Claude.dev72

    Claude Code cloud sessions: a field guide to running tasks on separate VMs

    Addy Osmani walks through Claude Code cloud sessions, where each task runs on a fresh Anthropic-managed VM with the repository cloned onto its own branch, and cloud sessions are included in Pro, Max, Team and Enterprise plans at no extra compute charge. In a four-session test on a sample Node repository, three sessions started within 16 seconds of each other finished 87 seconds after the first began, and the flaky-test session ran the suite 40 times with zero failures. The guide covers setup scripts, network levels, GitHub connection paths and seven workflows suited to cloud sessions, while noting that local sessions remain the better fit for tasks needing local data, VPN services, GPUs or Zero Data Retention.

    Vì sao đáng chú ý: A hands-on field guide to Claude Code cloud sessions, showing how per-task VMs enable parallel work and where local sessions still fit better.

1 tháng 10Thứ Năm
  1. Anthropic · Claude.dev60

    Getting started with Claude Code mods: building a context-window forecast

    Claude Code mods are small JavaScript or TypeScript files that run inside a session as hooks shipped in plugins, and can observe events, rewrite behavior or draw custom UI in the terminal or desktop app. The guide builds Token Weather, an roughly 80-line mod that reads $.session.usage() after each turn and draws a weather-style context-window band above the prompt, then tours Blast Radius, which holds risky Bash commands, and Replay Theater, which steps through a turn's edits. Mods require Claude Code 2.1.287 or later and are on by default; finished code for all three is in anthropics/claude-code-playground.

    Vì sao đáng chú ý: A hands-on walkthrough of Claude Code mods that builds a working context-window meter in about 80 lines, showing how hooks, state and custom UI fit together.