Sing VL S1, a 4B-parameter multimodal decision model built on Qwen3.5, now has a GGUF release for llama.cpp CPU inference. It ships in BF16 (8.42 GB), Q8_0 (4.48 GB) and Q4_K_M (2.71 GB) decoders plus a shared F16 vision projector, with a default 4096-token context. GGUF validation covers 9 examples and 11 questions; full public scores for the GGUF variants are still pending.
Amazon Bedrock's cache duration, service tier, reasoning effort, and maxTokens settings can change inference cost without altering the model or prompt. A 5-minute cache write on Claude Sonnet 5 and Claude Opus 5 costs 1.25x the input rate, a 1-hour write costs 2x, and cache reads cost 0.1x; Flex tier requests to unsupported models fail with a ValidationException.
Google DeepMind introduced Gemini 4 Hydrogen, a Gemini 4 model built for fast everyday coding and agentic execution rather than the long-horizon reasoning of Gemini 4 Carbon and Argon. It scores 51 on the Artificial Analysis Intelligence Index at its default high thinking level versus 50 for Argon, reaches 54 at a new xhigh setting, and is priced flat at $1.50 per million input tokens and $7.50 per million output tokens across its 1 million-token context window. Google also launched Orchestrator in Google Antigravity, which switches between models like Carbon and Hydrogen within a session and reached near-Carbon task success on Terminal-Bench 4.0 while using 24% less quota and finishing 19% faster.
Why it matters: Google's Gemini 4 Hydrogen targets fast everyday coding and agentic work, with flat $1.50/$7.50 per million token pricing across a 1M context window and a new Orchestrator that switches between Gemini 4 models.
OpenRouter published a break-even analysis showing when self-hosting an open-weight LLM costs less than calling it through an API, using per-provider prices as of October 8, 2026. At those prices, Qwen3.8 27B on one rented RTX 3090 at $0.50 an hour breaks even at 6 to 12% load, while DeepSeek V4.1 Flash on four H200s never beats the cheapest zero-data-retention endpoint. The post also notes that provider.zdr and the EU and US domains cover most privacy needs without self-hosting, and recommends running a steady base load locally while sending peaks to the API.
vLLM supports Thinking Machines' Inkling 2 Small on day 0, a multimodal text-and-vision MoE with 594B total and 17B active parameters and up to 1M tokens of context. The integration adds kernel fusion with PDL, sequence parallelism for prefill that cuts an 8K-token prefill step by 22% at TP4 and 26% at TP8, and a Gluon relative attention decode kernel that drops a batch-1 decode step at 100K context from 15.1 ms to 4.8 ms. vLLM reports up to 630 TPS/user at TP4 on GB200 with acceptance length 3.5, and also supports LoRA and RL via SkyRL.
Why it matters: vLLM 在发布当天即支持 Inkling 2 Small，并给出内核融合、序列并行和 Gluon 解码内核的具体优化数据，可供部署大 MoE 模型的团队参考。
vLLM reports serving DeepSeek-V4.1-Flash on a single NVIDIA Vera Rubin NVL72 rack at 1.7M tokens/s per GPU with 387 tokens/s per user, 58x the GB300 NVL72, peaking at 2.52M tokens/s and 1,810 tokens/s for a single user. The gains come from kernels rebuilt for Rubin's split memory domains and FP4 tensor cores, a specialized TP8 decode path, and rack-wide KV and Engram pools shared over NVLink. With prefixes pooled across the rack and Dynamo workload-aware routing, the prefix hit rate holds at 96.5-97% even at high concurrency.
Why it matters: vLLM's third DeepSeek-V4.1-Flash post details how Rubin-specific kernels and rack-wide memory pooling lift throughput 58x over GB300 NVL72, offering a concrete reference for large-scale serving optimization.
An open source engine computes semantic diffs of formal policies using anti-unification over an e-graph, showing shared structure once and keeping both versions of each differing part. It targets autoformalization outputs in SMT-LIB, Cedar, Dogwood, OpenShell policies and CEL, where an LLM writes the artifact but plays no part in computing the diff.
AWS will hold its annual re:Invent conference from November 30 to December 4, with the AI/ML track offering more than 140 unique sessions out of over 2,000 total learning sessions. The guide maps sessions to the AWS AI stack, spanning AI chips (AWS Trainium and NVIDIA GPUs), models and inference on Amazon Bedrock and SageMaker AI, and agent development with Amazon Bedrock AgentCore. Sessions are organized from newcomer content to technical deep dives, and many can be viewed on demand.
Google introduced the Gemini Decisions API, powered by the new Gemini Decision 1 model (gemini-decision-1) and available through the Gemini API in Google AI Studio. A single multimodal input of up to 256K tokens can be checked against up to 1,000 yes/no, multiple-choice and rubric-scoring questions in one forward pass, returning probabilities and confidence scores at 150-250 ms end-to-end latency for $0.10 per 1M input tokens with no output-token charges. Google says the model leads its benchmark suite with the lowest calibration error and is nearly 5x cheaper than Gemini 3.5 Flash-Lite with the Interactions API; internal uses include model routing in Gemini Notebook, wireframe generation in Stitch and adversarial-content checks for MCP tools in Search.
Why it matters: Google's new endpoint scores up to 1,000 multimodal yes/no, choice and rubric questions in one pass at 150-250 ms, giving agent routing and policy checks a cheaper alternative to generative calls.
Claude Opus 5.5 is now the default model in Claude on Pro, Max, Ultra and Team plans, and Anthropic recommends it as the daily driver for research, analysis and finished documents, decks and spreadsheets. Anthropic says Opus 5.5 at Medium effort matched or beat Opus 5 at High on its coding and knowledge-work evaluations, and in an internal merger-analysis test it finished in 63 minutes at half the cost of Opus 5 on the same task. The post also lists prompting changes for the new model, including dropping 'think hard' instructions, starting at Medium effort and attaching charts instead of retyping their values.
Why it matters: Anthropic's own guide explains why Opus 5.5 is now the default for daily knowledge work and which prompting habits to drop for it.
inclusionAI released LemonDecider 4B v3, a decision model fine-tuned from Qwen3.5-4B that returns choice, yes/no or ordinal probabilities via a single joint forward pass instead of generating answer tokens. It was trained on about 1.20M decision/QA rows and scored 61.67 on the public Decision Index 0.4 suite, with median request latency of 49.9 ms.
inclusionAI released SingVL S1, a 4B multimodal decision model built on Qwen3.5 that answers named questions over text or JSON and up to four images in a single Transformers forward pass. It supports choice, yes/no (noul) and 2–10 level score questions, scoring 73.94 on the Public Vision Index and 69.76 on the Public Text Index. The model ships with Python inference code and a FastAPI HTTP server.
Google DeepMind made Gemini 4 Carbon available to developers, enterprises, scientists and consumers, positioning it above Gemini 4 Argon for tasks that need deeper reasoning and more tokens. Carbon ranks #1 on CursorBench 4.0, Perplexity's WANDR and Zapier's AutomationBench at more than 2X lower cost per task than the next best external model, and is priced at $2 per million input tokens and $10 per million output tokens, the same introductory price as Argon. Google also previewed /swarm in Google Antigravity, which turns a single prompt into a coordinated team of agents, and reported that Carbon blocked 99% of attack trajectories in simulated cyberattack testing.
Why it matters: Google's release notes detail how Gemini 4 Carbon trades higher token usage for deeper reasoning, ranking first on CursorBench 4.0, WANDR and AutomationBench at under half the per-task cost of the next external model.
OpenAI researchers examined how pretraining, architecture, reinforcement learning and model-generated data affect two proxies for chain-of-thought monitorability: no-CoT capability and CoT controllability. They report that no-CoT capability tracks lower base-model perplexity smoothly, that CoT controllability rises during RL even without process supervision beyond length constraints and final-answer grading, and that controllability can be passed to a student model through SFT on a teacher's rollouts. The authors caution that both proxies are imperfect and that models may become harder to monitor through their CoT as capabilities scale.
Why it matters: OpenAI's lifecycle study links lower pretraining perplexity and RL scaling to weaker CoT monitorability, offering a concrete frame for how training choices shape oversight risk.
Hugging Face published an explainer on decision models, which take a state plus typed questions and return probabilities over predefined answers instead of generating text. The post details the choice, score and noul primitives, the confidence formula, and new support through Transformers' DecisionPipeline and AutoModelForDecision, a shared /v1/systemone endpoint in Transformers and llama.cpp, a Hub tag for discovery, and fine-tuning with the Trainer API. It notes that the same endpoint also works with compatible autoregressive LLMs by scoring answer logits in a batched forward pass.
Why it matters: Hugging Face explains how decision models turn classification into a typed, non-generative interface and shows how to serve and fine-tune them with Transformers and llama.cpp.
Apple researchers and Harvard collaborators propose weighted state souping, a method that learns query-dependent weights for recombining cached SSM state encodings, improving long-context performance on single- and multi-hop document QA, needle-in-a-haystack retrieval, variable tracking, and tool use. A sparse variant achieves near-constant encoding cost with respect to context length while preserving or sometimes improving performance. The paper was accepted at the PALM workshop at NeurIPS 2026.
An Apple and Harvard study shows that the gating mechanism in State Space Models (SSMs) causes them to first learn an in-weights "memorization" solution, delaying or even preventing convergence to correct in-context learning. This occurs even when the architecture and memory capacity impose no fundamental limits. Gating, however, often improves generalization to long sequence lengths.
Goodfire Research published a study arguing that individual MLP neurons in Qwen3.6-35B-A3B are more interpretable than assumed, based on sparsification, attribution tools and research agents. Ablating the 92.2% weakest MoE neuron activations per forward pass raised language model loss by only 2.9%, and 69% of neuron poles had a single mode responsible for at least 90% of their causal effect. Natural-language neuron descriptions let a judge model predict firing with 95% accuracy against random text and recover 73% of ablation damage through simulation.
Why it matters: Goodfire's study of Qwen3.6-35B-A3B neurons argues that strong activations alone carry most of the model's computation, offering a concrete alternative to feature-dictionary interpretability.
Researchers introduce HOPE, a second-order expert pruning method for Mixture-of-Experts language models that accounts for pairwise interactions between co-activated experts instead of scoring each expert independently. At 50% pruning, HOPE achieves the best average rank of any method, with gains of up to +6.1% on agentic coding tasks over the next-best pruning method. The work targets the memory bottleneck in models such as Qwen3.5-122B, where expert MLPs take up 95% of a ~250 GB footprint.
Preferred Networks released PLaMo 3.0 Fin, a finance-specialized small language model built on the PLaMo 3 NICT pretraining model from joint research with NICT and further trained on Japanese financial data, publishing open weights on Hugging Face under the PLaMo Community License. The model adds step-by-step Reasoning with reasoning traces output in Japanese, plus tool calling for web search, calculation and internal database access, and can run on a single GPU in closed cloud or on-premise environments. PFN says the instruction-following training for Reasoning and tool use was carried out on its own MN-Core 2 AI chip, with code porting completed within a week and training speed comparable to a GPU environment.
NVIDIA Research and Inferact's REINFORCE Zero paper shows that aligning rollout and trainer kernels lets unclipped token-level importance sampling correct the remaining weight lag, training stably for 20,000 updates at policy lag 64 and reaching 2.40x the training-step throughput of synchronous training with aligned kernels. With standard kernels, the median rollout-to-learner KL rises 83.33x across the run and online AIME24 pass@1 falls from 34.27% to 13.96%, while a sequence gate stabilizes training but rejects 24.48% of sequences by the last 1,000 updates. The zero-mismatch setup now covers DeepSeek-V4.1-Flash, Kimi K3.1 on release day, and DeepSeek-V4-Flash and Kimi K3.1 on AMD MI355X, with code in Inferact/zero-mismatch-rl and NVIDIA-NeMo/labs-molt.
Why it matters: NVIDIA Research and Inferact split the async RL rollout-learner gap into weight lag and kernel mismatch, showing that aligning kernels lets unclipped importance sampling hold stable training at policy lag 64.
Zyphra Research released ZYFA, BF16 and FP8 causal prefill attention kernels for AMD's MI355X (CDNA4, gfx950) supporting head dimension 128 and DeepSeek-V2's MLA shape. On activations from Mistral-7B, Qwen3-8B and DeepSeek-V2-Lite at 16K tokens, ZYFA's FP8 kernels reach 1.16–1.76× AITER's throughput, and on Gaussian inputs at 32K tokens 1.26–1.37× at head dimension 128 and 1.99× at the MLA shape. FP8 error is 62–78% lower than AITER's at head dimension 128.
ASpec is a high-throughput training framework for speculative decoding draft models, built from the TorchSpec codebase with optimized attention, fused kernels and selective head/loss computation. On a Qwen3-8B 4K run, Draft-only training reaches 15,256.44 input tokens/s on one GPU, 1.96× TorchSpec, while the 8-GPU online steady window hits 77,721.37 tokens/s, 2.28× TorchSpec. It supports DSpark, DFlash, EAGLE3/4 and native MTP drafts, with Qwen3-8B and Ling tiny quickstarts.
Goodfire Research proposes mechanistic shaping, a framework that adds penalties on a model's internal concept directions during training so it learns to rely on or express specific concepts rather than just maximize task reward. In chest X-ray experiments, an LLM agent used sparse autoencoder features to find tube-like shortcut directions, and reliance shaping beat both standard training and a balanced-data baseline.
Buoyant announced Kinference, an Apache v2 open source control plane that routes OpenAI- and Anthropic-compatible inference traffic to vLLM workers on Kubernetes, adding token rate limiting, per-tenant usage and cost tracking, KV-cache-aware load balancing, and YAML inference policies. In the project's public benchmarks, Kinference cut median time-to-first-token by 95% and median end-to-end latency by 45% versus standard Kubernetes service load balancing on one workload, and served 7.4x the goodput on shared-prefix and about 3.5x on multiturn in a capacity test. Policies can route simpler prompts to cheaper models, burst overflow to providers such as Bedrock and Databricks, or keep PII-tagged requests inside the cluster, and per-tenant metrics cover token rates, TTFT, ITL and cached versus uncached prompt tokens.
Why it matters: Buoyant's Kinference adds KV-cache-aware load balancing and YAML inference policies on top of vLLM, with benchmarks claiming large TTFT and goodput gains over standard Kubernetes service routing.
Ai2 introduced StateBench, a benchmark with four difficulty levels for long-context state tracking, from immutable assignment to self-referential and cross-referential updates. In tests, Gemini 3.8 Flash, Claude Opus 5.5, and GPT-6 Sol and Luna handled the easier levels but dropped to 13% or lower on the longest general finite problems, while 275M-parameter Gated DeltaNet and hybrid models trained from scratch generalized almost perfectly on the first three levels and transformers never learned the hardest one.
Apple researchers propose a two-stage recipe for distilling a Transformer into a pure Mamba architecture without any Attention blocks. The method first distills a pretrained Transformer into a linearized Attention model via an adapted kernel trick, then distills that into an adapted Mamba model. The resulting distilled Mamba retains most of Pythia-1B's performance, reaching 14.01 perplexity versus the teacher's 13.86, with some gaps on tasks relying on in-context recall.
Google ran a 50-question auto-graded coding-agent benchmark testing whether a lightweight classifier, TypeSafe's Jev, can route each task to the cheapest Gemini configuration that still answers correctly. A frozen regex and prompt-length heuristic scored 93.8% on the 16 development questions but dropped 23.2 points to 70.6% on 34 held-out tasks, while Jev reached 96.0% across the full suite. The strongest no-router setup, Gemini 3.8 Flash on low thinking, solved all 50 tasks, and a Jev router sending tasks to Flash-Lite first and escalating flagged ones also solved all 50 while reducing cost and response time; capping maxOutputTokens at 300 saved nothing on easy tasks and broke 7 of 25 hard ones.
Zyphra introduced Zyphra Hierarchical Attention (ZHA), a sparse attention method that builds a tree of compressed KV-cache summaries and routes queries top-down through it, giving subquadratic complexity with suitable top-p thresholds. Existing pretrained models can be adapted via a short KL-distillation phase using only a few billion tokens; on Qwen3.6-35B-A3B, ZHA adds 81,920 trainable parameters, about 0.00024% of the base model. On RULER-13 at 256K tokens the wide-tree model scored 96.2% versus 96.0% for dense attention, and the deep six-level tree scored 63.4% on LongBench v2 versus 58.0% for dense while attending to about 1.1% of the KV cache; a GPU roofline model projects 32-35x speedup over dense attention at 10M tokens.
Anthropic released FlashProve, an open-source prototype that uses zkSNARKs to prove a response came from a committed model without revealing its weights. On Llama 4 Maverick (400B-parameter MoE, 4-bit weights), a single H100 generates proofs at about 110 tokens/second with ~19 MB proofs per 1,024-token request, roughly 70-95x the GPU time vLLM needs for the same work. The release includes executable prover and verifier code, a C++ fast verifier, a Lean verifier with machine-checked completeness, soundness and zero-knowledge statements, and a technical report, all under Apache 2.0.
Why it matters: Anthropic's FlashProve shows zkSNARK-based proofs of inference are becoming practical for a 400B-parameter MoE model, with open code and Lean-checked security claims.
Sakana AI held its "From World Models to Real-World AI" symposium in Tokyo on October 26, 2026, where Chief Scientific Advisor Jürgen Schmidhuber gave a keynote and Sakana AI announced it will develop its own pretrained language model series, Sakana Sawara (tentative name). The first small model, Sawara Small (tentative name), is planned for release in spring 2027 if development proceeds smoothly. More than 400 attendees joined the event, which also featured a conversation between Schmidhuber and CEO David Ha.
Combining knowledge distillation with the Medusa framework lets a 1.1B-parameter TinyLlama student match the quality of its 70B-parameter Llama-3.1 teacher while running 10 to 20 times as fast at less than one-third the cost. Working with Booking.com on AI Trip Planner, Smart Filters and accommodation description generation, the pipeline closed over 90% of the quality gap and added 2 to 3.6x further speedup from Medusa heads.
Sakana AI applied its narrative analysis system to roughly 710,000 reactions on X to posts from both candidates and state party organizations in the Texas Senate race, estimating support and criticism ratios with statistical error margins. Its Sakana Marlin product cross-checked SNS observations against polling, campaign finance, prediction markets and news, finding shifts in candidate evaluation about two months before major raters like Cook Political Report changed their assessments.
inclusionAI released Ling-3.0-flash-Sante-fp8, an FP8-quantized version of its healthcare-specialized Ling-3.0-flash-Sante model, keeping 124B total and 5.1B active parameters per token. It matches the BF16 checkpoint's 73.9 average across healthcare benchmarks while cutting weight memory for FP8-capable hardware. The research-only model supports 256K-token contexts and defaults to thinking mode with tool calling.
Meta detailed the next generation of Andromeda, its personalized ads retrieval engine, which scores over one billion ads per second in production — a 100x throughput lift over the 2024 system. The new architecture brings LLM-inspired mixture-of-experts to retrieval, approaching ranking-level model performance at roughly 300x fewer inference-time FLOPs, with over 10 billion ads per second on a single NVIDIA GH200 GPU. Since 2025, Andromeda has delivered a 2.4% increase in conversions on Facebook.
The UBTree project page presents UBTree, a speculative decoding method that drafts parallel token trees using unigram and bigram models, detailed in arXiv:2609.39972. Demo videos compare UBTree against AR, DFlash and DSpark on Ling3 Tiny and Flash using unchanged HumanEval anti_shuffle and bf prompts, with all four methods producing identical output tokens. The site is plain HTML/CSS/JS served via GitHub Pages, with chart and table numbers stored in static/js/data.js.
Google Research introduced a sleep framework for continual learning that alternates active interaction with offline memory consolidation, letting LLMs absorb new facts into expanded Mixture-of-Experts parameters instead of losing prior knowledge. The approach combines parameter expansion, on-policy distillation with reinforcement learning-based imitation, and a dreaming stage that generates and filters synthetic training samples. The authors report near-zero performance drop in continual translation for unseen languages such as Manchu and Kalamang, near-perfect scaling to 10M tokens on BABILong, 48.9% single-task and 46.2% CPT results on SQuAD-based factual incorporation, and 79.2%, 69.0% and 46.1% on AIME 24/25 and HMMT 25.
Apple announced a new iMac powered by the M6 chip, featuring an up-to-12-core CPU and GPU with Neural Accelerators in each core, plus a Dual 16-core Neural Engine for up to 5x faster on-device AI than M4. Apple claims up to 18x faster performance than the most popular Intel-based iMac and up to 12x faster AI prompt processing in LM Studio versus M1. Pre-orders open October 28, with availability from Friday, November 6.
Apple announced the new 14-inch MacBook Pro with M6, featuring a 12-core CPU and 12-core GPU that deliver up to 30 percent faster graphics and up to 25 percent faster AI workloads than M5. The chip adds a Dual 16-core Neural Engine and GPU Neural Accelerators for on-device AI, with up to 24 hours of battery life, Wi-Fi 7, and Bluetooth 6. Pre-orders open October 28, with availability starting Friday, November 6.
Apple researchers present GHARP, a method that animates 3D human heads in real time from a few input images plus a driving expression signal, splitting the task into an offline identity stage and a lightweight runtime animation stage. A body alignment network removes ambiguity from body pose and clothing, and GHARP reaches state-of-the-art quality on the Ava-256 benchmark while running up to 13× faster on an A100 GPU with 8× fewer Gaussians.