Skip to content
Only Research#Coding
TodayNovember 13Friday2 items
  1. Hacker News · AI58

    Study finds AI-style stale version pins in 13.7% of new npm packages in 2026 H1

    Kobi Mozgori sampled 250,000 npm packages and found that new packages whose pinned dependency versions are mostly round x.0.0 numbers already superseded rose from a 1–2.4% baseline in 2019 to early 2024 to 13.7% in 2026 H1, falling to 7.6% in 2026 H2 through early November. The median pinned dependency was about 155 days old at birth in 2021–2023 but 451 days in 2026 Q2, and the share of packages born with essentially all pins on the newest release dropped from about 28% to about 10%. The author frames the pattern as an inference rather than proof, since registry metadata cannot show who or what wrote a manifest, and notes the decline since mid-2026 is unexplained.

  2. Hacker News · AI58

    How much of GitHub's most-starred AI tooling is written by coding agents?

    An analysis of the full git history of 52 of GitHub's most-starred AI-tooling repositories, covering 324,088 commits, finds that in the median repo 16.9% of commits in its last 180 days of activity carry an AI signature, and 21 of 52 sit at 25% or more. Stars and AI-signed share show no detectable link (Spearman ρ = -0.05, p = 0.72), and 85.3% of the 36,892 signed commits come from Claude, whose tool adds a trailer by default. The author cautions that heavy AI use is concentrated in projects started in 2025 or later, so the sample cannot separate AI use from recency, and every AI percentage is a lower bound.

November 6Friday
  1. Microsoft Research58

    Microsoft Research's PDD Kit formally verifies agent-written full-stack web apps

    Microsoft Research introduced PDD Kit, a research prototype that turns a developer's plain-language intent into a Verus formal specification, proves the coding agent's Rust code meets it for every input, and falls back to tests where a proof cannot reach. In a demo inside GitHub Next's Chopin, the team used PDD Kit to verify the login logic of Pet Store, a roughly 5,000-line Rust full-stack app, with the developer reviewing the intent artifact rather than the generated code line by line. PDD Kit only verifies Rust today, and its guarantees rest on six developer-approved trusted pieces, including the bcrypt password check and the two database lookups.

October 30Friday
  1. Meta Engineering Blog58

    Meta details production deployment of Catching JiTTests with human-on-the-loop review

    Meta has moved Catching JiTTests, its LLM-based just-in-time test generation approach, from research prototype into production infrastructure, and published the results in the FSE 2026 paper "Just-in-Time Catching Test Generation at Meta." The system runs overnight on spare capacity, targets risky diffs, generates tests through dodgy-diff and intent-aware workflows, and filters false positives with an LLM-as-Judge ensemble plus a rule-based RubFake assessor before reaching engineers. As of writing, 589 engineers (64% of outreach) engaged with the results, which showed a 60% true-positive rate, and in the first half of 2026 the system caught 74 serious privacy bugs at the diff stage and lowered privacy SEVs by 20%.

October 24Saturday
  1. Epoch AI · Gradient Updates42

    Epoch Brief: AI Work Automation Test, SpireClimb Benchmark, and Gemini 4 Argon's Frontier Push

    Epoch AI tested six leading models on 11 realistic Epoch work tasks and found none could reliably meet its quality bar, with Claude Fable 5.1 and GPT-6 Astra roughly tied for the lead. Its new SpireClimb benchmark shows frontier models went from struggling at Slay the Spire to winning over 90% of base-difficulty runs in six months, with GPT-6 Astra at 98%. Gemini 4 Argon scored 166 on the Epoch Capabilities Index, tying Claude Fable 5.1 and trailing Claude Opus 5.5 and GPT-6 Astra at 169.

October 5Monday
  1. GitHub Blog · AI & ML58

    GitHub releases ReviewBench, an open benchmark for AI code review

    GitHub released ReviewBench, an open benchmark for AI code review agents built on 219 public pull requests across 19 languages, with language and repository-size distributions modeled on 103.9 million GitHub pull requests. The golden set draws findings from human reviewers, author follow-up commits, static analysis and multiple frontier LLMs, graded by Claude Sonnet 5 under a published rubric, and senior engineers independently re-labeled every finding with 96.6% agreement. GitHub says its offline evaluation of Copilot code review anticipated the direction of a production A/B test, where addressed rate rose 8.0%, recall rose 13.6% and cost per review fell 8.0%.

April 29Wednesday
  1. Factory58

    Factory benchmarks 13 models on code review, finds GPT-5.2 best and MiniMax M2.7 cheapest

    Factory built an open-source benchmark testing 13 models on 50 real pull requests from Sentry, Grafana, Keycloak, Discourse and Cal.com, with each model reviewing every PR at least three times at high reasoning effort and an LLM judge scoring findings against a human-curated golden set. GPT-5.2 led at 60.5% F1 for $1.25 per PR, ahead of Claude Opus 4.6 at 59.8% F1 and $3.11 per PR, while MiniMax M2.7 reached 45.6% F1 at $0.15 per PR and Kimi K2.5 hit 51.9% F1 at $0.41. Factory says cost explains only about 21% of quality variance, and it open-sourced the golden set, evaluation scripts and a Droid Action GitHub Action supporting all 13 models.