Do I need an OpenAPI / Swagger spec?

No. Validate.QA derives API tests entirely from the traffic your app generates during exploration, so it covers what the app actually does — including undocumented endpoints and the real sequences your UI depends on. If you have a spec, it isn't required and isn't relied on.

Can it test the API without driving the UI?

Yes. Generated tests use Playwright's request fixture, which makes HTTP calls directly with no browser or page. They run headless and fast, on their own, and can be executed independently of the UI suite.

How is authentication handled?

Auth is detected from the captured traffic. If endpoints used a Bearer token, tests log in first, extract the token from the response, and pass it as an Authorization header on later calls. If the app uses session cookies, tests just call login first and let Playwright's request context carry the cookie automatically. Login credentials come from environment variables, so no secrets are baked into the test files.

What does each test actually assert?

Every test asserts the HTTP status and at least one response-body property taken from the observed shape — a field name and the value you expect. Happy-path tests assert success codes and key fields; validation tests assert specific error codes like 401, 404, or 400. A test never mixes success and error codes in one permissive check, so a passing test means something real.

How does it handle stateful flows where one call needs an ID from another?

It replays calls in the order they originally happened and threads values between them: the token from authentication and the ID returned by a create carry into the read, update, and delete steps. These run as a serial, shared-state chain so the dependencies hold end to end, with IDs pulled from responses rather than guessed.

What happens when a generated test fails — is it just rewritten to pass?

No. Failures are triaged per sub-test. If the test itself was wrong — an outdated status, a renamed field, the wrong auth format — it's corrected automatically. If the API itself is broken (a 500 on valid input, an auth bypass, an endpoint that now 404s), it's reported as an application bug and marked as a known issue instead of being weakened to go green.

API testing

Validate.QA doesn't need an OpenAPI spec to test your API. As the agent explores your app, it captures the calls your UI actually makes, learns each endpoint's payload shape and auth, then writes Playwright API tests that chain them in the right order.

Mechanism: Sequence-aware generation from captured network traffic: normalized endpoints and request/response shapes drive Playwright request-fixture tests with status and body assertions..

The problem: API tests are usually written by hand, separately from UI tests, and quietly drift out of sync as the backend changes. Without seeing real traffic you end up guessing at payload fields, headers, and how auth is actually carried. Spec-driven generators only cover what's documented — they miss undocumented endpoints and the real call sequences your app depends on. Stateful flows (create then read then delete) need IDs and tokens threaded between requests, which static fixtures rarely model correctly.

Observe real traffic — While the agent explores your UI, every request and response that flows through the page is captured — method, path, headers, status, and body. URLs under /api/, GraphQL endpoints, and JSON responses are flagged as API calls, and auth is detected from the live request: a Bearer Authorization header is recorded as header auth, a session/JWT cookie as cookie auth. This is real traffic from real flows, never a guess from a schema.

Normalize endpoints — Captured paths are normalized into stable patterns: UUIDs, numeric IDs, MongoDB ObjectIds, and CUID/nanoid segments collapse to :id, so /api/orders/4850 and /api/orders/clf2x9a0z1 both map to GET /api/orders/:id. Calls are grouped by method and pattern and merged across runs into a project-level endpoint map that tracks how often each endpoint was seen, which success and error statuses it returned, and whether it requires auth.

Extract payload shapes — For every call, the JSON request and response bodies are reduced to a field-level shape — key names and value types, including nested objects and arrays — rather than storing raw data. Response shapes are kept per status code and merged across observations, so generated tests can assert on fields the API genuinely returns instead of properties someone hoped were there.

Sequence into dependency chains — API calls are read back in the exact chronological order they happened during exploration, which exposes their dependencies: authenticate, create a resource, read it back, delete it. Tests are built as a shared-state chain where the token from login and the ID from a POST carry into the GET, PUT, and DELETE that follow — IDs are extracted from responses, never hardcoded blindly.

Generate request-fixture tests — Tests are emitted as native Playwright specs using the request fixture (no browser, no page). The stateful happy-path chain runs as test.describe.serial so steps execute in order and share state; independent validation cases (401, 404, 400) run as a plain test.describe so one failure never skips the rest. Each test pairs a status assertion with a body assertion drawn from the captured response shape — status-only checks are not enough.

Run, verify, and self-heal — Specs run headless in the same suite and CI as your UI tests, fast because there's no browser to drive. When a test fails, the system classifies each sub-test: a test-side mistake (wrong status, changed field name, bad auth format) is fixed automatically, while a genuine backend defect — a 500 on valid input, an auth bypass, a 404 on a known endpoint — is flagged as an app bug and quarantined rather than silently rewritten to pass.

Open API testing · Get Started Free