Self-Healing Playwright Tests: How AI Fixes Flaky Tests Automatically
Flaky tests cost teams hours every week. Learn how AI-powered self-healing detects broken selectors and test steps, then rewrites them automatically.
A designer renames a button class, and a dozen end-to-end tests go red while the feature still works. Maintenance like this is the usual reason generated and hand-written suites get abandoned. Self-healing in Validate.QA repairs that kind of failure automatically, and it is built so it does not hide real bugs.
Why tests break
Most broken tests come from a few causes:
Retries do not help with the first kind: a wrong selector fails on every attempt. Re-running only hides intermittent failures. Fixing the test code is what helps, and doing that by hand for every redesign is the cost self-healing removes.
How a failed run is handled
Tests run as ordinary Playwright code, with no AI in the loop. AI is involved only after a run fails.
1. Decide whose fault it is
Validate.QA reads the failure, the screenshot and the captured network traffic and sorts it into one of three groups: test drift (a selector or step moved), an environment problem (rate limit, quota, lost sign-in), or a real app regression. Environment problems are tagged and skipped, not quarantined, and the tag clears on the next clean pass. A failure that looks like a real app bug is not repaired. The test is quarantined and flagged as a found bug. Only drift goes on to healing.
2. Repair the code
The healer first tries a script-level fix: it re-locates the moved element and rewrites the broken step. If that is not enough, it opens the failing page in a real browser, finds the new path and writes the fix back into the test.
3. Prove it, then save it
The repaired test runs again. Only a fix that passes is saved, and the previous version of the test is kept so you can see what changed. If the repair does not hold, the test is not rewritten into something that merely passes. It is left for review, tagged @flaky.
Will self-healing hide real bugs?
It is designed not to. Healing changes how a test finds and uses an element, not what the test expects. If a button is renamed from Delete to Remove, the healer updates the locator. If the button now does nothing, the assertion on the outcome still fails, and that failure is classified as an app bug and quarantined instead of being repaired. A test quarantined for an app bug stays quarantined until you re-enable it after the bug is fixed, so a lucky green run does not close it. Tests quarantined for other reasons return to active after two passes in a row.
Topics: Self-Healing, Flaky Tests, AI.
Read the full article · Get Started Free