Playwright AWS CodeBuild Setup

Stream Playwright results from AWS CodeBuild to TestDino live, with sharded build passes grouped into one run.

Set up an AWS CodeBuild project that streams Playwright test results to TestDino during the run and view aggregated analytics, failure analysis, and flaky test detection on your dashboard.

Prerequisites

• A TestDino account with a project created
• A TestDino API key (Generate API Keys)
• An AWS account with CodeBuild access
• A Playwright test project (or clone the TestDino Example Repository to get started)
• @testdino/playwright installed and configured in playwright.config.ts|js|mjs:

npm install @testdino/playwright

playwright.config.ts

import { defineConfig } from '@playwright/test';

export default defineConfig({
  reporter: [
    ['@testdino/playwright', {
      token: process.env.TESTDINO_TOKEN,
      serverUrl: "https://reporter.testdino.com",
    }],
  ],
});

Set Up Your API Key

Store your TestDino API key as a CodeBuild environment variable so it is available to your build without exposing it in buildspec files.

1. Open your AWS CodeBuild project
2. Choose Edit
3. Open the Environment section
4. Add an environment variable named TESTDINO_TOKEN
5. Paste your TestDino API key as the value
6. Save the project

Warning

Never commit your API key directly in buildspec files. Always use CodeBuild environment variables. For additional security, store the key in AWS Systems Manager Parameter Store or AWS Secrets Manager and reference it in your environment configuration.

Basic Buildspec Config

For a simple setup without sharding, run your tests with TESTDINO_TOKEN set. Results stream to TestDino during the run, so no separate upload step is needed.

buildspec.yml: Basic buildspec

buildspec.yml

version: 0.2

phases:
  install:
    runtime-versions:
      nodejs: 20
    commands:
      - npm ci
      - npx playwright install --with-deps
  build:
    commands:
      - npx playwright test

Tip

Set TESTDINO_TOKEN in the CodeBuild project environment so @testdino/playwright reads it during the run. Results stream live even when tests fail, so failures land on the dashboard without a post_build upload phase.

Sharded Test Runs

For larger test suites, CodeBuild runs Playwright across multiple shard passes within a single build. Each shard streams its own results, and TestDino groups them into one run when they share a ci-run-id.

How it works

1. CodeBuild runs 4 Playwright shard passes sequentially in a single build
2. Each shard streams its results to TestDino during the run
3. Every shard passes the same --ci-run-id so TestDino merges them into a single run
4. No merge or upload step runs after the shards finish

Full sharded config

buildspec.yml: Sharded buildspec

buildspec.yml

version: 0.2

phases:
  install:
    runtime-versions:
      nodejs: 20
    commands:
      - npm ci
      - npx playwright install --with-deps
  build:
    commands:
      - |
        for SHARD in 1 2 3 4; do
          npx tdpw test --ci-run-id "$CODEBUILD_BUILD_ID" -- --shard=$SHARD/4
        done

Key details

Config Block	What It Does
runtime-versions: nodejs: 20	Uses Node.js 20 managed runtime
for SHARD in 1 2 3 4	Runs 4 shard passes sequentially within a single build
--shard=$SHARD/4	Passes the shard value directly to Playwright
--ci-run-id "$CODEBUILD_BUILD_ID"	Groups all shard passes into one run on the dashboard
TESTDINO_TOKEN	Set in the project environment, read by @testdino/playwright to stream results
TESTDINO_SERVER_URL	Set in the project environment to https://reporter.testdino.com, or keep serverUrl in the playwright.config reporter options.

Set the ciRunId reporter option in playwright.config to $CODEBUILD_BUILD_ID if you prefer running npx playwright test directly instead of npx tdpw test.

Pipeline execution

After the pipeline runs, CodeBuild logs show the TestDino output in CloudWatch under /aws/codebuild/<project-name>. Each shard streams its results, and TestDino merges them into one run.

Results in TestDino

The test run appears in your TestDino dashboard with full failure details, flaky detection, and trend data as tests complete.

Troubleshooting

Results not appearing on the dashboard

Confirm @testdino/playwright is in your playwright.config reporter array and TESTDINO_TOKEN is set in the project environment. Results stream during the run, so no separate upload step is required.

Sharded runs show up as separate runs

Pass the same --ci-run-id "$CODEBUILD_BUILD_ID" (or ciRunId reporter option) to every shard pass. Different values create one run per shard.

TESTDINO_TOKEN not available in build

Confirm the environment variable is set in your CodeBuild project under Environment → Environment variables. If using Parameter Store or Secrets Manager, verify the IAM role has read access to the secret.

Related

CI Optimization

Reduce CI time with smart reruns

Environment Mapping

Route test results to Dev, Staging, or Production by branch

Integrations

Connect Slack, Jira, Linear, Asana, and more

TestDino MCP

Access test results and fix issues with AI agents