Google AI Releases Auto-Diagnose: An Large Language Model LLM-Based System to Diagnose Integration Test Failures at Scale
If you have ever stared at thousands of lines of integration test logs wondering which of the sixteen log files actually contains your bug, you are not alone — and Google now has data to prove it. A team of Google researchers introduced Auto-Diagnose , an LLM-powered tool that automatically reads the failure logs from a broken integration test, finds the root cause, and posts a concise diagnosis directly into the code review where the failure showed up. On a manual evaluation of 71 real-world failures spanning 39 distinct teams , the tool correctly identified the root cause 90.14% of the time . It has run on 52,635 distinct failing tests across 224,782 executions on 91,130 code changes authored by 22,962 distinct developers , with a ‘Not helpful’ rate of just 5.8% on the feedback received. https://ift.tt/TCLkE2I The problem: integration tests are a debugging tax Integration tests verify that multiple components of a distributed system actually communicate to each other corre...

