Agentic testing

Mobile CI artifacts

Use this when Matcha should run saved Android or iOS scenarios against the fresh build from a pull request. This is part of mobile QA.

Matcha reads GitHub Actions artifacts from the repository bound in QA settings, and only uses artifacts built from the current PR head commit. Coldtea shows you the artifact name, workflow run, commit SHA, file name, size, and digest, so you can confirm it picked up the build you expected.

A run does not need your desktop. Once Coldtea reports it as started or queued, you can close the app and the run finishes without you. For a manual upload, keep the app open until the file finishes uploading and the run starts.

Configure Matcha first

In Coldtea settings, open QA, then GitHub PR testing, then Fresh mobile build QA.

Set:

  • The mobile platforms to run.
  • The scenario group mapping for each target branch.
  • The build wait timeout and retry interval.
  • One exact artifact name or one constrained one-star glob per line.

Keep the artifact names narrow. Matcha rejects regex-like patterns, path segments, broad globs, unsupported extensions, and ambiguous multiple matches.

Artifact names

Upload one matching artifact per platform per workflow run. The default settings recognize either default name, but uploading both defaults in the same run is ambiguous unless you narrow the setting to one name.

PlatformDefault artifact namesArtifact contents
Androidmatcha-android-apk or matcha-android-buildOne Android .apk file. Extra manifest text files are OK.
iOS Simulatormatcha-ios-simulator-app or matcha-ios-simulator-buildOne archive file, *.zip or *.tar.gz, containing an iOS Simulator .app bundle. Extra manifest text files are OK.

Custom names can omit a file extension like the defaults do. If a custom Android artifact name includes a terminal extension, use .apk or .zip. If a custom iOS artifact name includes one, use .zip or .tar.gz. The artifact name is not a path.

Android GitHub Actions example

This example uploads the default Android artifact and includes a small SHA manifest for human provenance. Matcha still validates the GitHub workflow run head SHA before using the artifact.

name: Android PR build

on:
  pull_request:
    branches: [main]

jobs:
  android:
    runs-on: ubuntu-latest
    env:
      MATCHA_HEAD_SHA: ${{ github.event.pull_request.head.sha || github.sha }}
    steps:
      - uses: actions/checkout@v4
        with:
          ref: ${{ env.MATCHA_HEAD_SHA }}

      - uses: actions/setup-java@v4
        with:
          distribution: temurin
          java-version: "17"

      - name: Build debug APK
        run: ./gradlew :app:assembleDebug

      - name: Prepare Matcha artifact
        run: |
          mkdir -p build/matcha
          cp app/build/outputs/apk/debug/app-debug.apk \
            "build/matcha/app-debug-${MATCHA_HEAD_SHA}.apk"
          printf '%s\n' "${MATCHA_HEAD_SHA}" > build/matcha/head-sha.txt

      - name: Upload Matcha Android artifact
        uses: actions/upload-artifact@v4
        with:
          name: matcha-android-apk
          path: |
            build/matcha/app-debug-${{ env.MATCHA_HEAD_SHA }}.apk
            build/matcha/head-sha.txt
          if-no-files-found: error
          retention-days: 7

The uploaded artifact must contain exactly one APK. Do not upload .aab, .xapk, multiple APK variants, or a broad build directory for this workflow.

iOS Simulator GitHub Actions example

Matcha uses iOS Simulator builds only. Build with the iphonesimulator SDK and upload a .zip or .tar.gz that contains the .app bundle. Do not export or upload an .ipa.

name: iOS Simulator PR build

on:
  pull_request:
    branches: [main]

jobs:
  ios-simulator:
    runs-on: macos-latest
    env:
      MATCHA_HEAD_SHA: ${{ github.event.pull_request.head.sha || github.sha }}
      SCHEME: MyApp
    steps:
      - uses: actions/checkout@v4
        with:
          ref: ${{ env.MATCHA_HEAD_SHA }}

      - name: Build Simulator app
        run: |
          xcodebuild \
            -scheme "${SCHEME}" \
            -configuration Debug \
            -sdk iphonesimulator \
            -destination 'generic/platform=iOS Simulator' \
            -derivedDataPath build/DerivedData \
            CODE_SIGNING_ALLOWED=NO \
            build

      - name: Package Simulator .app for Matcha
        run: |
          mkdir -p build/matcha
          APP_PATH="$(find build/DerivedData/Build/Products/Debug-iphonesimulator \
            -maxdepth 1 -type d -name '*.app' | head -n 1)"
          test -n "${APP_PATH}"
          ditto -c -k --keepParent "${APP_PATH}" build/matcha/Matcha.app.zip
          printf '%s\n' "${MATCHA_HEAD_SHA}" > build/matcha/head-sha.txt

      - name: Upload Matcha iOS Simulator artifact
        uses: actions/upload-artifact@v4
        with:
          name: matcha-ios-simulator-app
          path: |
            build/matcha/Matcha.app.zip
            build/matcha/head-sha.txt
          if-no-files-found: error
          retention-days: 7

A .tar.gz package is also supported:

tar -czf build/matcha/Matcha.app.tar.gz \
  -C "$(dirname "${APP_PATH}")" \
  "$(basename "${APP_PATH}")"

The archive must contain one Simulator .app bundle. Matcha rejects .ipa, disguised .ipa.zip and .ipa.tar.gz, missing .app bundles, and artifacts containing multiple app archives. Device builds from the iphoneos SDK are unsupported and out of scope. Rebuild with iphonesimulator before uploading.

Troubleshooting

What Coldtea showsWhat it meansWhat to check
Stale artifact SHAThe matching artifact belongs to an older PR commit, so Matcha will not use it.Rerun CI after the latest push. Make sure your checkout and build steps use the PR head SHA and the workflow ran for the current head.
Artifact not foundCI finished without producing an artifact that matches your configured name.Check the upload-artifact name, whether the platform is enabled, the branch and group mapping, whether the workflow succeeded, and whether the name is too broad or wrong.
Expired artifactGitHub no longer lets Matcha download the artifact.Rerun the workflow, or raise retention-days so the artifact outlives the build wait window and your review.
Permission deniedMatcha cannot reach the repository or its artifacts.Confirm the TeaHouse, the repo binding, the GitHub App installation, repository access, and Actions artifact permissions.
Provider registration failedMatcha found the build but could not get it onto a device.Confirm the artifact is valid, and that you have not exhausted your device quota or concurrency. Retry while the build is still available.
Build no longer availableThe build or the artifact it came from has aged out.Use whichever rerun option Coldtea offers. If none are available, rerun CI to publish a fresh artifact.
Rerun unavailableThe rerun you asked for depends on a build or artifact that has since expired.Pick a different rerun option if one is shown, or rerun the workflow for the current commit.

If the setup state is unclear, start with the GitHub workflow run for the current PR head SHA, then compare the uploaded artifact name and contents against the settings in Coldtea.

On this page