Andrew proposes Transcript Testing for verifying AI-generated code
Start here
Andrew, a blog author, proposes "Transcript Testing": instead of asserting on individual values, a test records a human-readable transcript of system behavior and checks it against an expected transcript, so a reviewer can read what the code actually did. He demonstrates it on an Apache DataFusion pull request about Parquet Bloom filter pruning, where a 400-line AI-generated test was replaced by transcript output showing which filters and pages were read. Andrew notes the approach is not new, citing Oracle's ORATEST and DataFusion's sqllogictests, but argues that AI now makes transcript generators cheap enough to use far more often.
Generated from reports · updated 59 min ago
Report timeline
Follow the reports to see the event from different sides.
- Hacker News · AITranscript Testing: Writing AI-Generated Code Tests for Human Review
Andrew proposes Transcript Testing, a testing style that records a human-readable transcript of system behavior and asserts it against an expected transcript, to help reviewers verify AI-written code. He illustrates it with an Apache DataFusion PR on Parquet Bloom filter pruning, where a 400-line AI-generated test was replaced by transcript output showing which filters and pages were read. He notes the approach is not new, citing Oracle's ORATEST and DataFusion's sqllogictests, but argues AI now makes transcript generators cheap enough to use far more often.
Heat for this event
Not enough continuous observations yet to draw a trend.