Tới nội dung

Lĩnh vực

Kỹ thuật triển khai Mới nhất

Chạy mô hình nhanh hơn và rẻ hơn: tăng tốc suy luận, bộ nhớ và chi phí, phục vụ mô hình và lựa chọn hạ tầng.

62 tin chọn34 trong 30 ngày quaĐã thu thập 472

Cập nhật 13 tháng 11 08:32

Kỹ thuật triển khai · Tuyển chọn · trang 3

1 tháng 10Thứ NămMục 41–60 trong 62
  1. Modal Blog60

    Modal launches VM Sandboxes, giving agents full Linux VMs with Docker support

    Modal has made VM Sandboxes generally available, giving agents a full Linux VM through the same APIs, modal.Images, sub-second cold starts and CPU/memory bursting as its existing gVisor-based Sandboxes. Users enable it with runtime="vm" when they need Docker, FUSE filesystems or niche Linux kernel features, while gVisor remains the default runtime. Modal says early customers have already launched over 20 million VMs, with Linear running every Coding Session on a VM Sandbox and Legora and Snorkel using them for legal agent evals and agent simulations.

    Vì sao đáng chú ý: Modal's VM Sandboxes give agents a full Linux machine with Docker and kernel access while keeping the existing Sandbox APIs and sub-second cold starts.

30 tháng 9Thứ Tư
  1. Factory62

    Factory makes Automations generally available for Droid agents

    Factory has made Automations generally available, letting teams describe a recurring workflow and have Droid run it on a schedule or event trigger. Each automation combines a trigger (schedule, Slack, GitHub, or webhook), instructions, a user-chosen model and reasoning level, and a machine that can be local, connected, or a Factory-managed Droid Computer. Factory says 52% of enterprise users ran automated code review in the past 30 days, 48% used security review, 35% used AutoWiki, and automation session volume grew 51% over the prior 30 days.

    Vì sao đáng chú ý: Factory makes its Automations generally available, showing which recurring engineering workflows enterprises actually hand to Droid and how model and machine choice work per task.

29 tháng 9Thứ Ba
  1. OpenClaw Blog71

    OpenClaw Enterprise launches as an open source platform for managing persistent agents

    OpenClaw announced OpenClaw Enterprise (OCE), an open source, vendor neutral platform for managing persistent agents in sensitive environments, currently in development ahead of its 1.0 release later this year. OCE adds an enterprise control plane with multi-tenancy, hard security boundaries, sandboxing, LLM-based reviews and fine-grained permissions, and lets the harness, model and sandbox be swapped for third party or internal implementations. Organizations can self host it today via docker-compose for local development or kubernetes for internal deployment, and it is being piloted at companies including Red Hat and OpenAI.

    Vì sao đáng chú ý: OpenClaw Enterprise is an open source control plane for persistent agents in sensitive environments, with multi-tenancy, hard security boundaries and auditability, now available for internal pilot workloads.

28 tháng 9Thứ Hai
  1. Anthropic · Claude.dev72

    Anthropic adds build-eval and hillclimb commands to the claude-api skill in Claude Code

    Anthropic added two commands to the claude-api skill inside Claude Code: /claude-api build-eval builds an evaluation set in your codebase, and /claude-api hillclimb iterates on an application against that evaluation with a held-out set to catch overfitting.

    Vì sao đáng chú ý: Anthropic adds build-eval and hillclimb commands to its claude-api skill, showing how Claude Code can design evaluations and iterate against them while guarding against overfitting.

  2. Anthropic · Claude.dev82

    Anthropic releases Claude Sonnet 5.5 with unchanged per-token pricing and 30% faster output

    Anthropic released Claude Sonnet 5.5, the second model in the Claude 5.5 family after Opus 5.5, which the company says is smarter, more efficient and 30% faster than Sonnet 5 at the same per-token price. Because it typically needs fewer tokens per task, Anthropic says most work costs up to 30% less, and the model ships with a native 1M-token context window, 128k max output and thinking on by default. The release guide lists breaking changes for Sonnet 5 users, including replacing disabled thinking with the between_tools setting, swapping forced tool_choice for auto plus strict tools, and moving computer use to the computer_toolset_20260801 toolset.

    Vì sao đáng chú ý: Anthropic's Sonnet 5.5 keeps Sonnet 5 pricing while cutting per-task token use, and the migration notes show what changes for teams already building on Sonnet 5.

22 tháng 9Thứ Ba
  1. OpenClaw Blog60

    OpenClaw adds opt-in decision models and a shared Plugin SDK interface

    OpenClaw introduced support for a separately configured decision model that its core and plugins can call through api.runtime.decisions.evaluate in the Plugin SDK, returning a choice, score or probability instead of a chat reply. The feature is opt-in and does not replace the chat model, and the provider packages are still awaiting a supporting release. The TypeSafe adapter supports hosted Jev and a local System One server such as Jared Palmer's Kev, and the evaluation tool is moving into OpenClaw core as decision_evaluate.

    Vì sao đáng chú ý: OpenClaw adds an opt-in decision model layer that returns typed answers from application code, with a shared Plugin SDK interface so plugins can use it without per-provider integrations.

12 tháng 8Thứ Tư
  1. Modal Blog62

    Qwen3.8-2.4T-A95B 开源权重发布，Modal 同步上线共享端点

    Qwen3.8-2.4T-A95B 已作为开源权重模型发布，并同步在 Modal 上线。Modal 表示其与 Qwen 在发布前合作，通过 SGLang 和针对 Qwen3.8 形状调优的定制 DFlash 推测器，为 Modal Auto Endpoints 提供 day zero 支持。

    Vì sao đáng chú ý: Modal 在 Qwen3.8-2.4T-A95B 开源权重发布当天即上线共享端点，并说明其定制 DFlash 推测器如何针对该模型的形状提升推理速度。

27 tháng 7Thứ Hai
  1. Modal Blog75

    Moonshot's Kimi K3 now available on Modal with day-zero vLLM support

    Modal made Moonshot's Kimi K3 available on its platform on release day, offering the 2.8 trillion parameter multimodal model with a 1M token context window through a token-priced Shared API and dedicated Auto Endpoints. Modal says a custom DFlash speculator tuned to K3's architecture raises interactivity from 100 to 460 tokens per second and throughput from 800k to 1.5 million TPM per GPU on agentic workloads. The model uses a mixture-of-experts design with 16 of 896 experts active per token, and Moonshot contributed a new prefix-caching implementation to vLLM ahead of the release.

    Vì sao đáng chú ý: Modal details how it brought Moonshot's 2.8T-parameter Kimi K3 to its platform on release day, including a custom DFlash speculator that lifts interactivity from 100 to 460 tokens per second.

16 tháng 7Thứ Năm
  1. Modal Blog60

    Modal rebuilds its sandbox platform to run 1 million concurrent sandboxes, created in under a minute

    Modal has rebuilt its core sandbox platform from the ground up, and on the new system it ran 1 million sandboxes concurrently, creating all of them in under a minute. The redesign removes central coordination and any data store from the sandbox creation path: horizontally scaled scheduling servers pick a worker with an in-memory algorithm and call it directly, while workers publish state asynchronously to a Redis stream. Modal says sandbox start latency is under half a second at the median, and the new system is already available in Beta with a one-line code change to opt in.

    Vì sao đáng chú ý: Modal rebuilt its sandbox control plane to drop central coordination, showing how far container scheduling can scale once global consistency leaves the critical path.

15 tháng 7Thứ Tư
  1. Modal Blog60

    Thinking Machines' Inkling multimodal model now available on Modal

    Thinking Machines released Inkling, a general-purpose multimodal model that takes text, image and audio inputs and outputs text, and Modal is offering it from day zero as a Managed Endpoint with token-based pricing. Inkling is a mixture-of-experts transformer with 975B total parameters, 41B active, a 1M token context window and native audio and vision, using five sliding-window attention layers for every full-attention layer. On agentic workloads with 8x B200s, Modal's custom DFlash speculator reached 250 tokens per second per user at 2.5M TPM of per-GPU throughput, 67% faster interactivity than the model's built-in speculative path at matched throughput.

    Vì sao đáng chú ý: Modal's day-zero endpoint for Thinking Machines' 975B-parameter Inkling shows how a custom DFlash speculator lifts interactivity 67% over the model's built-in speculative path.

30 tháng 6Thứ Ba
  1. Modal Blog66

    Anthropic launches Claude Science with Modal as scalable compute backend

    Anthropic launched Claude Science on June 30, an AI workbench where life sciences researchers run computational workloads directly inside a conversation with Claude. Modal integrates as the compute backend: workloads needing GPUs or many concurrent CPUs route automatically to Modal sandboxes from a researcher's connected workspace, while default execution stays sandboxed on the local machine. Modal cites fan-out virtual screening across hundreds of containers, per-step GPU access for models like ESMFold2, shared Modal Volumes storage and reproducible Modal Images, and commits up to $100,000 in compute for Anthropic's AI for Science Claude Science Cohort at $500–$2,000 per project, with applications open through July 15, 2026.

    Vì sao đáng chú ý: Modal's post explains how Claude Science routes GPU and parallel workloads to Modal sandboxes, with concrete life-science examples and a compute grant for academic labs.

23 tháng 6Thứ Ba
  1. Modal Blog60

    Modal launches Auto Endpoints for self-serve, OpenAI API-compatible LLM inference

    Modal launched Auto Endpoints, a self-serve service that deploys production-grade LLM inference as an OpenAI API-compatible endpoint backed by a Modal App the user can see and control. Each supported model ships with a starting deployment tuned from Modal's own benchmarking, using engines like SGLang and the DFlash speculative decoding drafter from Z Lab, and users can inspect GPU choices, engine flags and inference metrics such as TTFT, ITL and speculative decoding acceptance length. Modal also released Modal Servers from beta for regionalized routing with 5ms overhead, and said it generates endpoint deployments with an internal agentic system still monitored by human engineers.

    Vì sao đáng chú ý: Modal's Auto Endpoints expose the serving stack, engine metrics and benchmark recipes that managed inference providers keep hidden, giving teams a self-serve path to owning open-model inference.

19 tháng 6Thứ Sáu
  1. Modal Blog60

    Modal releases DFlash speculators for Qwen 3.5 and 3.6, claiming 5-20% extra speedup

    Modal released a set of DFlash draft models for the Qwen series on Hugging Face, including Qwen 3.6 35B-A3B-DFlash and Qwen 3.5 variants from 4B to 122B-A10B, plus a DFlash speculator for Qwen 3.5 397B-A17B built with SGLang. The company says the new drafters add 5-20% speedup over existing DFlash baselines and can run Qwen 3.5 122B-A10B at over 1000 tok/s at concurrency 1 on a B200 node, versus 250 tok/s without speculation. The post argues speculative decoding delivers 2x-3x gains where kernel optimization yields small percentage points, and presents a roofline model for estimating speedup from acceptance length.

    Vì sao đáng chú ý: Modal explains why it bets on speculative decoding and releases DFlash draft models for the Qwen series, with measured speedups and a roofline model for estimating gains.

11 tháng 6Thứ Năm
  1. Factory62

    Factory rolls out automated security review in Droid

    Factory is rolling out automated security review in Droid, which runs a STRIDE-based security review alongside standard code review on every non-draft PR and posts findings with severity, a CWE reference, an explanation and a suggested fix as inline comments on the diff. The company says Droid's security review has been run across production codebases and surfaced findings that went through responsible disclosure, including CVE-2026-42876 in external-secrets and workos/workos-node#1578. Droid builds a lightweight threat model of the changed surface, scans for vulnerabilities such as OWASP Top 10 and OWASP LLM Top 10 issues, then validates each candidate finding against the diff to filter false positives before posting one deduplicated summary per PR.

    Vì sao đáng chú ý: Factory's Droid now runs a STRIDE-based security review on every non-draft PR, and the post shows real CVEs it surfaced in production codebases.

27 tháng 5Thứ Tư
  1. MiniMax Blog72

    MiniMax upgrades Agent as Mavis and launches Agent Teams for long-running tasks

    MiniMax upgraded MiniMax Agent under the new name Mavis and launched Agent Teams, which let multiple role-specific Agents run in parallel on the desktop for long, complex tasks. The system uses Leader, Worker and Verifier roles with a state-machine Team Engine that cycles tasks through producing, verifying and done, and it merges TokenPlan and Agent Plan into one subscription spanning CLI, API and Agent across M2.7, music, video and voice. MiniMax says the Agent will be open-sourced soon, expected alongside the MiniMax M3 model, while the desktop app is already available.

    Vì sao đáng chú ý: MiniMax explains the design tradeoffs behind its Leader-Worker-Verifier Agent Team, including handoff, sharing and aggregation costs, offering a concrete reference for building multi-Agent runtimes.

17 tháng 4Thứ Sáu
  1. OpenAI Developers Blog60

    OpenAI publishes API deployment checklist for GPT-6 workloads

    OpenAI published an API deployment checklist that walks developers through moving workloads onto the Responses API and the GPT-6 model family, including gpt-6-astra, gpt-6.1-sol, gpt-6 Sol and gpt-6 Luna. The guide covers reasoning.effort levels from low to max, text.verbosity, tool_search with deferred tool loading, Programmatic Tool Calling, Multi-agent, compaction, prompt caching and WebSocket mode, which OpenAI says has shown up to roughly 40% faster end-to-end execution for workflows with 20 or more tool calls. It also details reliability and safety steps such as handling 429 and 503 errors, background=True for long jobs, safety_identifier and misalignment monitoring.

    Vì sao đáng chú ý: OpenAI's official deployment checklist consolidates model selection, reasoning effort, tool search, compaction and WebSocket mode into one migration guide for GPT-6 workloads.

11 tháng 2Thứ Tư
  1. Modal Blog78

    Modal 上线 GLM-5 免费端点并发布自部署与前端接入指南

    Modal 推出 GLM-5 的免费 OpenAI API 兼容端点，可用至 4 月底，限单个并发请求，同时发布在单节点 8 张 B200 上用 SGLang 部署的可复现代码。

    Vì sao đáng chú ý: Modal 为 Z.ai 的 GLM-5 提供免费 OpenAI 兼容端点，并给出在 OpenCode、OpenClaw、Claude Code 和 Vercel AI SDK 中接入的具体配置。

26 tháng 9, 2025Thứ Sáu
  1. Modal Blog60

    Modal engineers reverse-engineer how Flash Attention 4 works

    Modal engineers read the released Flash Attention 4 source code and reverse-engineered how the Blackwell-optimized attention kernel works, since no official technical report exists yet. The write-up traces a tile through the kernel's warp-specialized asynchronous pipeline and details two math tricks: a cubic-polynomial software exponential that can replace SFU-based exponentiation, and a smarter online softmax rescaling rule that Tri Dao said cuts corrections by a factor of 10. FA4 is reported to be roughly 20% faster than Nvidia's closed-source cudnn attention kernels, and a backwards version was not yet available at the time of writing.

    Vì sao đáng chú ý: Modal engineers read the released Flash Attention 4 source and explain its warp-specialized pipeline and two math tricks, offering a transferable view of where CUDA kernel design is heading.

30 tháng 7, 2025Thứ Tư
  1. Modal Blog60

    Modal launches GPU memory snapshots for sub-second serverless cold starts

    Modal announced GPU memory snapshots in alpha, extending its existing checkpoint/restore system to capture GPU vRAM, CUDA kernels, streams and contexts so GPU workloads restore from a frozen state instead of rebuilding CUDA state after boot.

    Vì sao đáng chú ý: Modal adds GPU state to its memory snapshots, cutting cold boots for torch.compile and vLLM workloads by up to 10x and making scale-to-zero more practical.

28 tháng 1, 2025Thứ Ba
  1. Modal Blog62

    Modal adds memory snapshots to cut serverless cold starts by 2.5x

    Modal introduced memory snapshot restore for its serverless GPU container runtime, which it says makes cold starts about 2.5x faster than standard container startup. A Stable Diffusion inference Function that normally takes around 13 seconds now restores in 3.5 seconds, and a simple import torch program drops from roughly 5 seconds to about 1.05 seconds at p50 and 0.69 seconds at p0. Snapshots are built on gVisor's checkpoint/restore and are created on demand, with automatic fallback to a standard container startup when a restore fails; GPU state is not yet saved, so it must be rebuilt after restore.

    Vì sao đáng chú ý: Modal explains how gVisor-based memory snapshots cut serverless cold starts, with concrete restore timings for a Stable Diffusion Function and a torch import example.