Re-run Only Failed Tests

Reduce CI time and cost by running only tests that failed in the previous run.

When a CI run fails, re-running the entire job wastes time on tests that have already passed. This workflow runs only the failed tests on subsequent attempts.

Example

Your test suite has 500 E2E tests.

• A full run takes 1 hour
• 50 tests fail and take about 6 minutes to run

Re-running the entire suite means executing 450 passing tests again. With this approach, only the 50 failed tests are re-run.

Result: You save approximately 54 minutes per re-run.

Scenario	Full suite rerun time	Failed only rerun time	Time saved	Cost saved on Linux	Cost saved on Windows	Cost saved on macOS
Per rerun	60 min	6 min	54 min	$0.324	$0.54	$3.348
Per day (10 reruns)	600 min	60 min	540 min (9 hrs)	$3.24	$5.40	$33.48
Per 30-day month (10 reruns daily)	18,000 min	1,800 min	16,200 min (270 hrs)	$97.20	$162.00	$1,004.40

Note: Pricing is based on GitHub’s official Actions runner pricing

for GitHub-hosted runners:

• Linux (2-core): $0.006 per minute
• Windows (2-core): $0.010 per minute
• macOS: $0.062 per minute

This guide covers:

• Sharded Playwright execution
• Re-run failed jobs in GitHub Actions
• Failed-tests-only re-runs when failure metadata exists
• Full-suite fallback when no test metadata exists

Quick Reference

Task	Command/Action	Link
Run full test suite	npx playwright test	-
Cache failed test metadata	npx tdpw cache	Setting up cache
Get last failed tests	npx tdpw last-failed	How it works
Re-run detection	Check github.run_attempt	Workflow logic
Upload HTML report	actions/upload-artifact@v4	Quick start

How it works

This workflow uses two TestDino CLI commands:

• npx tdpw cache stores failed test metadata after a run
• npx tdpw last-failed returns Playwright arguments for the last failed tests

The workflow detects re-runs using github.run_attempt:

Condition	Behavior
run_attempt = 1	Run full test suite
run_attempt > 1, flags found	Run failed tests only
run_attempt > 1, no flags	Run full test suite (fallback)

Playwright compatibility: Since @playwright/test@1.50+, Playwright has native --last-failed support. TestDino extends this with cross-runner caching, shard awareness, and workflow-level persistence.

Quick start

Add this logic to your Playwright step:

- name: Run Playwright with rerun logic
  env:
    TESTDINO_TOKEN: ${{ secrets.TESTDINO_TOKEN }}
  run: |
    echo "Run attempt: ${{ github.run_attempt }}"
    if [[ "${{ github.run_attempt }}" -gt 1 ]]; then
      npx tdpw last-failed > flags.txt
      FLAGS="$(cat flags.txt)"
      if [[ -n "$FLAGS" ]]; then
        echo "Re-running only failed tests"
        eval "npx playwright test $FLAGS"
        exit 0
      fi
    fi
    echo "Running full test suite"
    npx playwright test

- name: Upload HTML report
  uses: actions/upload-artifact@v4
  with:
    name: Playwright Test Report
    path: ./playwright-report
    retention-days: 14

- name: Cache failed test metadata
  if: always()
  env:
    TESTDINO_TOKEN: ${{ secrets.TESTDINO_TOKEN }}
  run: npx tdpw cache

The cache step runs with the if: always() condition, so failures are recorded even when tests fail.

Need to set up your API token? See Generate API Keys for instructions.

How to re-run failed tests in GitHub Actions?

1. Open the failed job

   In the GitHub Actions run, find the failed job for Playwright:

   1. Go to your GitHub repository’s Actions tab.
   2. Select the failed workflow run.
   3. Under Jobs, open the job where Playwright tests get executed & fail.

   This is the main job that executes Playwright and sharding.

   
2. Check the run attempt

   Inside the job, open the Playwright execution step. In the logs, you can see:

   • GitHub run attempt: <number>

   • Re-run detection output

     
3. Identify the failed tests

   Scroll to the end of the Playwright output. GitHub Actions show:

   • Total failed test count

   • Failed test titles

     
4. Confirm caching

   Open the Cache failed test metadata step. Look for:

   Cache data submitted successfully
   

   This confirms TestDino stored the failure metadata.

   
5. Confirm Re-run

   Re-run the failed pipeline and confirm the execution of only the last failed tests.

   • The workflow detects run_attempt > 1

   • tdpw last-failed retrieves the failed tests

   • Only those tests are executed

     

Example: Sharded test execution

First run: 12 tests across 3 shards. Two shards fail.

Shard	Tests	Result
Shard 1	4 tests	2 Passed, 2 Failed
Shard 2	4 tests	Passed
Shard 3	4 tests	3 Passed, 1 Failed

Re-run comparison:

Without TestDino

├── Shard 1: runs 4 tests
├── Shard 3: runs 4 tests
└── Total: 12 tests

Re-runs all tests in failed shards, including tests that already passed.

With TestDino

├── Shard 1: runs 2 failed tests
├── Shard 3: runs 1 failed test
└── Total: 3 tests

Re-runs only the 3 tests that failed, not all 8 tests in those shards.

Full workflow

For a complete workflow with sharding and report merging, see the example repository.

View full workflow YAML

File name: .github/workflows/playwright.yml

Repository: github.com/testdino-hq/playwright-sample-tests-javascript

name: Run Playwright tests

on:
  push:
  pull_request:
  schedule:
    - cron: '30 3 * * 1-5' # 11:00 AM IST
  workflow_dispatch:

jobs:
  run-tests:
    name: Run Playwright tests ${{ matrix.shardIndex }}/3
    runs-on: ubuntu-latest

    strategy:
      fail-fast: false
      matrix:
        shardIndex: [1, 2, 3]
        shardTotal: [3]

    steps:
      - uses: actions/checkout@v4

      # ✅ REQUIRED: Node 20
      - name: Setup Node.js 20.x
        uses: actions/setup-node@v3
        with:
          node-version: '20'

      - name: Create .env file
        run: |
          echo "USERNAME=${{ secrets.USERNAME }}" >> .env
          echo "PASSWORD=${{ secrets.PASSWORD }}" >> .env
          echo "NEW_PASSWORD=${{ secrets.NEW_PASSWORD }}" >> .env
          echo "FIRST_NAME=${{ secrets.FIRST_NAME }}" >> .env
          echo "STREET_NAME=${{ secrets.STREET_NAME }}" >> .env
          echo "CITY=${{ secrets.CITY }}" >> .env
          echo "STATE=${{ secrets.STATE }}" >> .env
          echo "COUNTRY=${{ secrets.COUNTRY }}" >> .env
          echo "ZIP_CODE=${{ secrets.ZIP_CODE }}" >> .env

      - name: Cache npm dependencies
        uses: actions/cache@v3
        with:
          path: ~/.npm
          key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}
          restore-keys: |
            ${{ runner.os }}-node-

      - name: Install deps + browsers
        run: |
          npm ci
          npx playwright install --with-deps chromium firefox webkit

      # ✅ FULL + RERUN LOGIC
      - name: Run Playwright (rerun failed tests if applicable)
        env:
          TESTDINO_TOKEN: ${{ secrets.TESTDINO_TOKEN }}
          SHARD_INDEX: ${{ matrix.shardIndex }}
          SHARD_TOTAL: ${{ matrix.shardTotal }}
        run: |
          echo "GitHub run attempt: ${{ github.run_attempt }}"

          if [[ "${{ github.run_attempt }}" -gt 1 ]]; then
            echo "Detected re-run. Checking failed test metadata from TestDino."

            npx tdpw last-failed \
              --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }} \
              > last-failed-flags.txt

            EXTRA_PW_FLAGS="$(cat last-failed-flags.txt)"

            if [[ -n "$EXTRA_PW_FLAGS" ]]; then
              echo "Running only failed tests for this shard:"
              echo "$EXTRA_PW_FLAGS"

              # IMPORTANT: JSON + BLOB BOTH REQUIRED
              # Ensure playwright-report directory exists
              mkdir -p ./playwright-report
              eval "npx playwright test $EXTRA_PW_FLAGS"
              exit 0
            fi

            echo "No failed test metadata found. Falling back to full shard."
          fi

          # First run (full shard)
          # Ensure playwright-report directory exists
          mkdir -p ./playwright-report
          npx playwright test \
            --grep="@chromium|@firefox|@webkit|@android|@ios" \
            --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }}

      - name: Upload blob report
        if: ${{ !cancelled() }}
        uses: actions/upload-artifact@v4
        with:
          name: blob-report-${{ matrix.shardIndex }}
          path: ./blob-report
          retention-days: 1

      # ✅ THIS WILL SHOW "Metadata cached successfully ✔" WHEN JSON EXISTS
      - name: Cache tdpw last failed metadata
        if: always()
        env:
          TESTDINO_TOKEN: ${{ secrets.TESTDINO_TOKEN }}
          SHARD_INDEX: ${{ matrix.shardIndex }}
          SHARD_TOTAL: ${{ matrix.shardTotal }}
        run: |
          npx tdpw cache --verbose

  merge-reports:
    name: Merge Reports
    needs: run-tests
    if: always()
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v4

      # ✅ Node 20 here as well
      - name: Setup Node.js 20.x
        uses: actions/setup-node@v3
        with:
          node-version: '20'

      - name: Cache npm dependencies
        uses: actions/cache@v3
        with:
          path: ~/.npm
          key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}
          restore-keys: |
            ${{ runner.os }}-node-

      - name: Install deps + browsers
        run: |
          npm ci
          npx playwright install --with-deps

      - name: Download all blob reports
        uses: actions/download-artifact@v4
        with:
          path: ./all-blob-reports
          pattern: blob-report-*
          merge-multiple: true

      - name: Merge HTML & JSON reports
        run: npx playwright merge-reports --config=playwright.config.js ./all-blob-reports

      - name: Upload combined report
        uses: actions/upload-artifact@v4
        with:
          name: Playwright Test Report
          path: ./playwright-report
          retention-days: 14

      - name: Send TestDino report
        run: |
          npx --yes tdpw ./playwright-report \
            --token="${{ secrets.TESTDINO_TOKEN }}" \
            --upload-html \
            --upload-traces \
            --verbose

How the workflow logic works

The workflow uses a conditional check based on github.run_attempt:

if [[ "${{ github.run_attempt }}" -gt 1 ]]; then
  # Re-run: get failed tests from TestDino
else
  # First run: execute full shard
fi

• On the first attempt (run_attempt == 1), the full shard is executed.
• On subsequent attempts, the workflow attempts to re-run only previously failed tests.

What `last-failed` returns

The tdpw last-failed command outputs test filters formatted for Playwright:

-g "Verify that a New User Can Successfully Complete the Journey from Registration to a Single Order Placement @chromium|test @chromium"

These flags can be passed directly to npx playwright test.

Why `eval` is used

The command uses eval to handle quoted arguments in the test filter:

eval "npx playwright test $EXTRA_PW_FLAGS"

This keeps the -g "pattern" quoting intact when passed to Playwright.

Edge cases

Pipeline fails before tests run

If a job fails during dependency installation or setup, no test metadata exists.

On re-run:

1. tdpw last-failed returns nothing
2. The workflow detects the empty result
3. The job runs the full shard

No manual action required. The workflow handles this case automatically:

if [[ -n "$EXTRA_PW_FLAGS" ]]; then
  # Failed tests found, run only those
  eval "npx playwright test $EXTRA_PW_FLAGS"
else
  echo "No failed test metadata found."
  echo "Running full shard instead."
fi

And then, it will run the normal execution:

mkdir -p ./playwright-report
npx playwright test \
  --grep="@chromium|@firefox|@webkit|@android|@ios" \
  --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }}

Skipped tests due to `--max-failures`

Playwright’s --max-failures option stops test execution after N failures. For example:

npx playwright test --max-failures=2

Playwright stops after 2 failures. Tests that did not run are not recorded.

On re-run, only the failed tests execute. Skipped tests are not included as of now.

Related

CI overview, GitHub Actions, status checks, and flaky tests.

CI Optimization Overview

Learn about CI optimization strategies

GitHub Actions

Set up Playwright tests in GitHub Actions

GitHub Status Checks

Configure PR status checks

Flaky Tests

Detect and fix flaky tests