Python CLI

Install and configure testdino to upload pytest-based Playwright reports to TestDino.

testdino is a Python CLI that uploads pytest-based Playwright reports, caches test metadata, and retrieves failed tests for reruns.

Quick Reference

Topic	Link
Installation	pip install testdino
Quick Start	Run tests, upload, cache
Commands	upload, cache, last-failed
Configuration	Environment variables
CI/CD	GitHub Actions, GitLab CI, Jenkins

Prerequisites

• Python >= 3.9
• pytest with pytest-playwright
• pytest-playwright-json (generates the required JSON report)
• TestDino API token (generate one)
• Git initialized repository (for commit and branch metadata)

Installation

pip install pytest-playwright-json pytest-html testdino

---

Quick Start

1. Run tests with JSON output

   pytest \
     --playwright-json=test-results/report.json \
     --html=test-results/index.html \
     --self-contained-html
   
2. Upload the report

   testdino upload ./test-results --token="your-token"
   
3. Cache metadata for reruns

   testdino cache --working-dir test-results --token="your-token"
   

The upload command requires a JSON report. Always run pytest with the --playwright-json flag.

---

Commands

upload

Upload Playwright test reports and artifacts to TestDino.

testdino upload <report-directory> --token="your-token"

Upload flags:

Flag	Description
--upload-images	Upload screenshots
--upload-videos	Upload video recordings
--upload-html	Upload HTML reports
--upload-traces	Upload trace files
--upload-files	Upload file attachments (.md, .pdf, .txt, .log)
--upload-full-json	Upload all attachments

# Upload with all artifacts
testdino upload ./test-results --token="your-token" --upload-full-json

# Upload with specific artifacts
testdino upload ./test-results --token="your-token" --upload-images --upload-videos

# Upload with environment tag
testdino upload ./test-results --token="your-token" --environment="staging"

cache

Store test execution metadata after a run. Powers the last-failed command.

testdino cache --working-dir test-results --token="your-token"

Option	Description	Default
--working-dir <path>	Directory containing test results	Current directory
--cache-id <value>	Custom cache ID override	Auto-detected

last-failed

Retrieve cached test failures for reruns. Outputs test identifiers that pass directly to pytest.

# Print failed tests
testdino last-failed --token="your-token"

# Rerun only failed tests
pytest $(testdino last-failed --token="your-token")

# Failed tests for a specific shard
testdino last-failed --shard="2/5" --token="your-token"

Option	Description	Default
--branch <value>	Branch name override	Auto-detected
--commit <value>	Commit hash override	Auto-detected
--shard <value>	Shard specification (e.g., 2/5)	None
--environment <value>	Environment name for filtering	None
--cache-id <value>	Custom cache ID override	Auto-detected

Global Options

These options apply to all commands.

Option	Description
-t, --token <value>	TestDino API token (required)
-v, --verbose	Enable verbose logging

---

Configuration

Set environment variables to avoid passing flags with each command.

Variable	Description
TESTDINO_TOKEN	Authentication token
TESTDINO_TARGET_ENV	Default environment tag

export TESTDINO_TOKEN="your-api-token"
export TESTDINO_TARGET_ENV="staging"

---

CI/CD Integration

GitHub Actions

.github/workflows/test.yml

name: Playwright Tests
on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0
      - uses: actions/setup-python@v5
        with:
          python-version: '3.11'

      - name: Install dependencies
        run: |
          pip install pytest pytest-playwright pytest-playwright-json pytest-html testdino
          playwright install chromium --with-deps

      - name: Run tests
        run: |
          pytest \
            --playwright-json=test-results/report.json \
            --html=test-results/index.html \
            --self-contained-html

      - name: Cache metadata
        if: always()
        run: testdino cache --working-dir test-results --token="${{ secrets.TESTDINO_TOKEN }}"

      - name: Upload reports
        if: always()
        env:
          TESTDINO_TOKEN: ${{ secrets.TESTDINO_TOKEN }}
        run: testdino upload ./test-results --token="${{ secrets.TESTDINO_TOKEN }}" --upload-full-json

GitHub Actions (Sharded)

.github/workflows/test-sharded.yml

name: Playwright Tests (Sharded)
on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        shardIndex: [1, 2, 3, 4]
        shardTotal: [4]
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0
      - uses: actions/setup-python@v5
        with:
          python-version: '3.11'

      - name: Install dependencies
        run: |
          pip install pytest pytest-playwright pytest-playwright-json pytest-html testdino
          playwright install chromium --with-deps

      - name: Run tests
        shell: bash
        env:
          TESTDINO_TOKEN: ${{ secrets.TESTDINO_TOKEN }}
        run: |
          mkdir -p test-results

          if [[ "${{ github.run_attempt }}" -gt 1 ]]; then
            testdino last-failed \
              --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }} \
              --token="$TESTDINO_TOKEN" > last-failed-flags.txt
            FAILED_TESTS="$(cat last-failed-flags.txt | tail -1)"

            if [[ -z "$FAILED_TESTS" ]]; then
              echo "No failed tests found. Exiting."
              exit 0
            fi

            eval "pytest $FAILED_TESTS \
              --playwright-json=test-results/report.json \
              --html=test-results/index.html \
              --self-contained-html -v" || true
            exit 0
          fi

          pytest \
            --playwright-json=test-results/report.json \
            --html=test-results/index.html \
            --self-contained-html -v || true

      - name: Cache metadata
        if: always()
        run: testdino cache --working-dir test-results --token="${{ secrets.TESTDINO_TOKEN }}"

      - name: Upload reports
        if: always()
        run: testdino upload ./test-results --token="${{ secrets.TESTDINO_TOKEN }}" --upload-full-json

GitLab CI

.gitlab-ci.yml

image: python:3.11

stages:
  - test

playwright-tests:
  stage: test
  script:
    - pip install pytest pytest-playwright pytest-playwright-json pytest-html testdino
    - playwright install chromium --with-deps
    - pytest --playwright-json=test-results/report.json --html=test-results/index.html --self-contained-html
    - testdino upload ./test-results --token="$TESTDINO_TOKEN" --upload-full-json
  when: always

Jenkins

Jenkinsfile

pipeline {
    agent any

    environment {
        TESTDINO_TOKEN = credentials('testdino-token')
    }

    stages {
        stage('Test') {
            steps {
                sh 'pip install pytest pytest-playwright pytest-playwright-json pytest-html testdino'
                sh 'playwright install chromium --with-deps'
                sh 'pytest --playwright-json=test-results/report.json --html=test-results/index.html --self-contained-html'
                sh 'testdino upload ./test-results --token="$TESTDINO_TOKEN" --upload-full-json'
            }
        }
    }
}

Related

Explore test results, optimize your CI pipeline, and manage API keys.

View Test Runs

Explore test results in the platform

CI Optimization

Optimize your CI pipeline

Node.js CLI

Use TestDino with Playwright for Node.js

Generate API Keys

Create and manage API tokens