TestDino Glossary

Definitions of TestDino and Playwright testing terms: flaky test, error grouping, trace viewer, failure classification, test execution, MCP, and more.

Definitions of the core terms used across TestDino and Playwright test reporting. Each entry gives a one-line definition, the key details, and a link to the feature that uses it.

Quick Reference

Term	One-line definition
Test run	One execution of a test suite that reports results to TestDino
Test case	A single test() or it() block
Test suite	A set of test cases grouped in a spec file or project
Flaky test	A test that passes and fails without code changes
Failure classification	The category assigned to a failure’s root cause
Error grouping	Clustering failures by shared error message
Trace viewer	Step-by-step replay of a test execution
Real-time streaming	Live upload of results while tests run
Test execution	One billed unit: a single test case attempt
Code coverage	Percentage of source code exercised by tests
Stability score	A test’s pass rate across its run history
MCP	Protocol that lets AI agents query TestDino
Quality gate	Rules that pass or fail a CI check
Environment mapping	Rules that tag runs by environment from the branch
Personal Access Token	A user-scoped credential for MCP and the API

Test run

One execution of a Playwright test suite that reports its results to TestDino.

• Carries metadata: branch, commit, CI provider, duration, pass/fail counts
• Links to the pull request that triggered it
• Tracked over time so you can compare runs across branches and pipelines

Explore the run view in Test Runs.

Test case

A single test() or it() block that runs and reports one result.

• Result is one of: pass, fail, or flaky
• The smallest unit TestDino tracks
• The unit used to measure usage

See how cases are tracked in Test Cases.

Test suite

A set of test cases grouped together, usually by spec file or Playwright project.

• Aggregates suite-level metrics: pass rate, duration, flakiness
• Helps locate the least stable areas of a codebase

Learn how suites fit together in Key Concepts.

Flaky test

A test that produces different results, passing in one run and failing in another, with no code change.

How TestDino detects it:

Scope	Signal
Within a single run	Fails, then passes on retry
Across runs	Inconsistent outcomes over time

Sub-categorized by root cause: Timing, Environment, Network, Assertion, or Other.

Read how detection works in Flaky Test Detection.

Failure classification

The category TestDino assigns to a failed test based on its likely root cause.

The four categories:

Category	Meaning
Actual Bug	A real defect in the application
UI Change	An expected change that broke an assertion
Unstable Test (Flaky)	Non-deterministic behavior, not a real failure
Miscellaneous	Everything else (config, environment, infra)

Turns a wall of red into a triage queue, so teams fix real bugs first.

See how categories are assigned in AI Failure Analysis.

Error grouping

Clustering of failed and flaky tests by their shared error message within a run.

• Review 5 underlying errors instead of 40 individual failures
• Each group shows the affected test count
• Each group links to every test case it covers

Learn how grouping works in Error Grouping.

Trace viewer

The step-by-step replay of a single test execution.

Shows, in sequence:

• Each action and assertion
• Network calls and console logs
• DOM snapshots and screenshots

TestDino renders the standard Playwright trace in-platform, so you debug without downloading artifacts.

See it in action in the Trace Viewer guide.

Real-time streaming

Results sent to TestDino live while the run is still in progress.

• Package: @testdino/playwright
• Trigger: add the reporter to playwright.config, then run npx playwright test
• No separate upload step: each test case streams as it finishes

Set it up in Real-Time Test Streaming.

Test execution

The billed unit of usage in TestDino: one attempt of one test case.

Counts as an execution	Does not count
Each test case attempt	Skipped tests
Each retry (2 retries = 3 executions)	Attachments (screenshots, videos, traces)

See how usage is metered in Billing & Pricing.

Code coverage

The percentage of application source code exercised by a test run, measured with Istanbul.

• Ingested alongside test results
• Shows which code paths your Playwright suite reaches

Set it up in Code Coverage.

Stability score

A test case’s pass rate across its run history.

• Formula: (Passed / Total Runs) x 100
• Quantifies how trustworthy a test is over time
• Scoped to the active branch

See per-test trends in Test Case History.

MCP (Model Context Protocol)

The open protocol that lets AI agents read TestDino data in natural language.

• Works with agents like Claude and Cursor
• An agent can query test runs, failures, and flaky tests, then reason about them in chat
• Optional: TestDino works fully through the dashboard, CLI, and integrations

Get started in the MCP Overview.

Quality gate

Rules that determine whether a TestDino CI check passes or fails on a pull request.

A gate can block a merge on:

• Failed tests
• Flaky tests
• A drop in pass rate

Configure gates in GitHub Status Checks.

Environment mapping

Rules that tag each test run with an environment based on its branch name.

• Example: main maps to production, staging maps to staging
• Filter analytics by environment
• Compare stability across deployment targets

Set up rules in Environment Mapping.

Personal Access Token (PAT)

A user-scoped credential used to authenticate the Public API and the MCP integration.

Token	Prefix	Scope	Used by
Personal Access Token	td_pat_	User	Public API, MCP
Project API Key	td_api_	Project	Streaming/CI uploads, Azure DevOps Extension

Create one in Generate API Keys.

Related

Key Concepts

How test cases, suites, and classification fields fit together

Getting Started

Set up TestDino and upload your first run

Flaky Test Detection

How TestDino finds and categorizes flaky tests

MCP Overview

Connect AI agents to your test data