Agentic testing

Site protection for QA

Some deployments sit behind a gate that loads before your app does: a Vercel preview protection page, HTTP Basic Auth on a staging host, or a WAF that only lets through requests carrying a specific header. Matcha cannot test what it cannot reach, so site protection lets you hand Matcha the credential that opens the deployment.

This is different from a test user, the run-as identity Matcha signs in with inside your app. Site protection opens the door; a test user walks through it. You can use both together.

You configure one method per project, from the Site protection button on the Matcha QA screen. Matcha also sends x-coldtea-qa: true on protected requests so your app or WAF can recognize and allowlist test traffic. To override that value, define a custom header named x-coldtea-qa.

Vercel protection bypass

Use this when your preview URLs are gated by Vercel Deployment Protection, so you hit a Vercel login page instead of your app.

To get the secret, open your project in the Vercel dashboard, then Settings, then Deployment Protection, then Protection Bypass for Automation, then Add Secret (or copy the existing one). It is a long random string.

Paste that string into the Vercel protection bypass field. On the first request to the deployment host, Matcha sends it as x-vercel-protection-bypass. Vercel then sets a first-party bypass cookie so the rest of the run stays authenticated. The secret is only ever sent to the deployment host in the run's target URL.

HTTP Basic Auth

Use this when the host shows a browser username and password prompt before your app loads, the classic 401 Basic Auth dialog on a staging site.

Enter the username and password your host expects. Matcha sends them as an Authorization: Basic header to get past the gate. This is site-level protection only. If your app has its own login form, use a test user for that.

Custom headers

Use this for anything header-based: a WAF bypass policy, a Cloudflare Access service-token header, or a "this is test traffic" tag your backend recognizes.

Add one or more header name and value pairs. Matcha sends them (host-scoped) on every request to the target host. Header names must be valid HTTP tokens, and values cannot contain line breaks. You can set up to 20 custom headers.

How it stays safe

  • Encrypted at rest. Coldtea encrypts secrets with a dedicated key and never returns them to the UI. You see only the last four characters, and you re-enter a secret to replace it.
  • Host-scoped. Coldtea sends the credential only to the host of the run's target URL. Requests to third-party hosts (OAuth providers, CDNs, analytics) never receive it.
  • Never in URLs or logs. The secret rides in request headers, not the target URL, and Coldtea never writes it to run logs or the report.

Set it up

  1. Open the Matcha QA screen for your project and click Site protection.
  2. Pick a method and enter its credential.
  3. Save. It applies automatically to every run for that project: manual, scheduled, and PR-triggered.
  4. Start a run against the protected URL. If it loads the app instead of the protection wall, you are set.

Notes and limits

  • The target URL's host must be the protected host. If you test a custom domain but the deployment is served from a different host (or a cross-host redirect), the bypass cookie may not apply. Point the run at the protected host.
  • If your app itself reads the Authorization header, prefer custom headers over HTTP Basic Auth so the two do not collide.
  • Site protection applies to web runs. Mobile app runs do not pass through these deployment gates.

On this page