Continuous integration

Both clients need the same two things from your CI, and nothing else:

1. A run id that every shard of one run agrees on. It is what makes eight parallel jobs join a single pipeline and split one allocation between them instead of eight jobs each claiming the whole suite.
2. Which shard this job is — N of M, one-based.

On GitLab both are detected automatically. Everywhere else you supply at least one of them.

What is detected, and what you pass

CI	Run id	Branch	Shard
GitLab CI	CI_PIPELINE_ID	CI_COMMIT_REF_NAME	CI_NODE_INDEX / CI_NODE_TOTAL
GitHub Actions	GITHUB_RUN_ID + GITHUB_RUN_ATTEMPT	GITHUB_HEAD_REF, GITHUB_REF_NAME	pass --shard=N/M
Anything else	set TESTFLAKE_RUN_ID	set TESTFLAKE_BRANCH	pass --shard=N/M

TESTFLAKE_RUN_ID and TESTFLAKE_BRANCH always win when set, so you can override the detected values on GitLab and GitHub too.

A run id is never invented for you

If the client cannot work one out it will not generate one. A per-process id would create a separate pipeline for every shard, each holding the whole suite — slower than not sharding at all. It warns on stderr and runs the whole suite instead, so a misconfigured pipeline is slow, never wrong.

Two details that catch people out on CI specifically:

• Shard numbers are one-based. GitLab’s CI_NODE_INDEX already is. CircleCI’s CIRCLE_NODE_INDEX is zero-based and is not read by either client — add one yourself, as the CircleCI tab below does.
• The run id must survive a retry. Re-running a single failed job has to produce the same id as its siblings, or that job starts a pipeline of its own. This is why the GitHub id includes GITHUB_RUN_ATTEMPT: a full re-run is a new run, while retrying one job of an existing run is not.

Recipes

Each tab is a complete, minimal pipeline. TESTFLAKE_KEY comes from your CI’s secret store in every one of them — it is a credential, so do not commit it.

GitLab CI

parallel: sets CI_NODE_INDEX and CI_NODE_TOTAL, and every parallel job of one pipeline shares CI_PIPELINE_ID. Nothing has to be passed — this is the only CI where the wrapper needs no arguments at all.

Add TESTFLAKE_KEY under Settings → CI/CD → Variables, masked.

Playwright

.gitlab-ci.yml

playwright:
    image: mcr.microsoft.com/playwright:v1.62.1-noble
    parallel: 8
    variables:
        TESTFLAKE_HOST: https://app.testflake.com
    script:
        - npm ci
        - npx testflake npx playwright test

PHPUnit

.gitlab-ci.yml

phpunit:
    image: php:8.3-cli
    parallel: 8
    variables:
        TESTFLAKE_HOST: https://app.testflake.com
    script:
        - composer install --no-interaction
        - vendor/bin/testflake vendor/bin/phpunit

GitHub Actions

The run id and branch are detected; the shard is not, because a matrix job has no idea it is one of several. strategy.job-total is the matrix size, so the denominator stays correct when you change the shard list.

fail-fast: false matters here: with the default, one failing shard cancels the others mid-run and their results are never reported, so the next run is balanced from partial data.

Add TESTFLAKE_KEY under Settings → Secrets and variables → Actions.

Playwright

.github/workflows/tests.yml

name: Tests
on: [push, pull_request]

jobs:
  playwright:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        shard: [1, 2, 3, 4, 5, 6, 7, 8]
    env:
      TESTFLAKE_KEY: ${{ secrets.TESTFLAKE_KEY }}
      TESTFLAKE_HOST: https://app.testflake.com
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 22
      - run: npm ci
      - run: npx playwright install --with-deps
      - run: npx testflake npx playwright test --shard=${{ matrix.shard }}/${{ strategy.job-total }}

PHPUnit

.github/workflows/tests.yml

name: Tests
on: [push, pull_request]

jobs:
  phpunit:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        shard: [1, 2, 3, 4, 5, 6, 7, 8]
    env:
      TESTFLAKE_KEY: ${{ secrets.TESTFLAKE_KEY }}
      TESTFLAKE_HOST: https://app.testflake.com
    steps:
      - uses: actions/checkout@v4
      - uses: shivammathur/setup-php@v2
        with:
          php-version: '8.3'
      - run: composer install --no-interaction
      - run: vendor/bin/testflake --shard=${{ matrix.shard }}/${{ strategy.job-total }} vendor/bin/phpunit

CircleCI

Nothing is detected here, and one thing actively misleads: CIRCLE_NODE_INDEX is zero-based, so shard 1 of 8 arrives as 0. Neither client reads it — add one yourself.

For the run id use CIRCLE_WORKFLOW_ID, which is shared by every container of a parallel job. CIRCLE_BUILD_NUM is not — it differs per container, and using it would give you eight one-shard pipelines.

Add TESTFLAKE_KEY as a project environment variable or in a context.

Playwright

.circleci/config.yml

version: 2.1

jobs:
  playwright:
    docker:
      - image: mcr.microsoft.com/playwright:v1.62.1-noble
    parallelism: 8
    environment:
      TESTFLAKE_HOST: https://app.testflake.com
    steps:
      - checkout
      - run: npm ci
      - run:
          name: Run tests
          command: |
            export TESTFLAKE_RUN_ID="$CIRCLE_WORKFLOW_ID"
            export TESTFLAKE_BRANCH="$CIRCLE_BRANCH"
            npx testflake npx playwright test \
              --shard=$((CIRCLE_NODE_INDEX + 1))/$CIRCLE_NODE_TOTAL

workflows:
  tests:
    jobs:
      - playwright

PHPUnit

.circleci/config.yml

version: 2.1

jobs:
  phpunit:
    docker:
      - image: cimg/php:8.3
    parallelism: 8
    environment:
      TESTFLAKE_HOST: https://app.testflake.com
    steps:
      - checkout
      - run: composer install --no-interaction
      - run:
          name: Run tests
          command: |
            export TESTFLAKE_RUN_ID="$CIRCLE_WORKFLOW_ID"
            export TESTFLAKE_BRANCH="$CIRCLE_BRANCH"
            vendor/bin/testflake \
              --shard=$((CIRCLE_NODE_INDEX + 1))/$CIRCLE_NODE_TOTAL \
              vendor/bin/phpunit

workflows:
  tests:
    jobs:
      - phpunit

Jenkins

Jenkins has no parallel-index variable — you write the fan-out yourself, so the shard number is whatever your loop says it is. BUILD_NUMBER is per build and shared by every branch of a parallel block, which makes it a good run id; scope it with JOB_NAME so two jobs cannot collide.

BRANCH_NAME only exists on multibranch pipelines, hence the fallback.

credentials('testflake-key') binds a Secret text credential and masks it in the console log.

Playwright

Jenkinsfile

pipeline {
    agent none

    environment {
        TESTFLAKE_HOST   = 'https://app.testflake.com'
        TESTFLAKE_KEY    = credentials('testflake-key')
        TESTFLAKE_RUN_ID = "${env.JOB_NAME}-${env.BUILD_NUMBER}"
        TESTFLAKE_BRANCH = "${env.BRANCH_NAME ?: env.GIT_BRANCH}"
    }

    stages {
        stage('Tests') {
            steps {
                script {
                    int total = 8
                    def shards = [:]

                    for (int i = 1; i <= total; i++) {
                        int shard = i
                        shards["shard-${shard}"] = {
                            node('linux') {
                                checkout scm
                                sh 'npm ci'
                                sh "npx testflake npx playwright test --shard=${shard}/${total}"
                            }
                        }
                    }

                    parallel shards
                }
            }
        }
    }
}

PHPUnit

Jenkinsfile

pipeline {
    agent none

    environment {
        TESTFLAKE_HOST   = 'https://app.testflake.com'
        TESTFLAKE_KEY    = credentials('testflake-key')
        TESTFLAKE_RUN_ID = "${env.JOB_NAME}-${env.BUILD_NUMBER}"
        TESTFLAKE_BRANCH = "${env.BRANCH_NAME ?: env.GIT_BRANCH}"
    }

    stages {
        stage('Tests') {
            steps {
                script {
                    int total = 8
                    def shards = [:]

                    for (int i = 1; i <= total; i++) {
                        int shard = i
                        shards["shard-${shard}"] = {
                            node('linux') {
                                checkout scm
                                sh 'composer install --no-interaction'
                                sh "vendor/bin/testflake --shard=${shard}/${total} vendor/bin/phpunit"
                            }
                        }
                    }

                    parallel shards
                }
            }
        }
    }
}

Travis CI

Travis builds a job per entry in env.jobs, so the shard number is just another environment variable. TRAVIS_BUILD_ID is shared by every job of one build, which is what a run id needs; TRAVIS_JOB_ID is not.

Add TESTFLAKE_KEY under Settings → Environment Variables, with Display value in build log off.

Playwright

.travis.yml

language: node_js
node_js: 22

env:
  global:
    - TESTFLAKE_HOST=https://app.testflake.com
    - SHARD_TOTAL=8
  jobs:
    - SHARD=1
    - SHARD=2
    - SHARD=3
    - SHARD=4
    - SHARD=5
    - SHARD=6
    - SHARD=7
    - SHARD=8

install:
  - npm ci
  - npx playwright install --with-deps

script:
  - export TESTFLAKE_RUN_ID="$TRAVIS_BUILD_ID"
  - export TESTFLAKE_BRANCH="$TRAVIS_BRANCH"
  - npx testflake npx playwright test --shard=$SHARD/$SHARD_TOTAL

PHPUnit

.travis.yml

language: php
php: '8.3'

env:
  global:
    - TESTFLAKE_HOST=https://app.testflake.com
    - SHARD_TOTAL=8
  jobs:
    - SHARD=1
    - SHARD=2
    - SHARD=3
    - SHARD=4
    - SHARD=5
    - SHARD=6
    - SHARD=7
    - SHARD=8

install:
  - composer install --no-interaction

script:
  - export TESTFLAKE_RUN_ID="$TRAVIS_BUILD_ID"
  - export TESTFLAKE_BRANCH="$TRAVIS_BRANCH"
  - vendor/bin/testflake --shard=$SHARD/$SHARD_TOTAL vendor/bin/phpunit

Checking it worked

1. Run the pipeline twice. The first run has no timing data and falls back to an alphabetical split, so a single run tells you nothing about balance.

2. Compare the shard durations of the second run. They should finish within a few percent of each other. If one shard is still much slower, it is probably a single test longer than the average shard — nothing can split below one test.

3. Check stderr for [testflake] warnings. Every failure path says so there, and every one of them means the suite ran unsharded.

Environment variables

Variable	Default	Purpose
TESTFLAKE_KEY	—	Attributes runs to a project. Treat as a credential.
TESTFLAKE_HOST	—	API base URL.
TESTFLAKE_RUN_ID	detected on GitLab and GitHub	Ties the shards of one run together.
TESTFLAKE_BRANCH	detected on GitLab and GitHub	Branch the run belongs to.

When TestFlake is unreachable

Your build still passes. Any failure — an unreachable host, a non-2xx response, a malformed body — makes the client warn on stderr and run the entire suite, unsharded, exactly as if the wrapper were not there. You get a slow build, never a red one, and the exit code is always your test runner’s own.

That is worth knowing before you debug a pipeline that suddenly got slower: a run taking eight times as long usually means eight shards each ran everything.