Cypress vs. Playwright: Mastering Test Execution with Tags & Filters

This article explores how to control test execution in Cypress and Playwright using tags and filters. Learn how to leverage these features to streamline your testing process and improve efficiency by running specific subsets of tests.

What's Inside: Cypress vs. Playwright Test Control (Part 2)

This guide dives into the world of test tags and filters in Cypress and Playwright, including practical plugins to enhance their functionality. Understanding these features is crucial for efficient test management in your projects.

Tags: Categorize and Conquer Your Tests

Tags are essential for efficient organization and management of tests. They allow you to categorize tests based on various criteria such as test type (smoke, regression), priority, or specific functionality. By using tags effectively, you can enhance the flexibility, readability, and maintainability of your testing framework.

Common Uses of Tags:

• Test Selection: Run specific subsets of tests based on certain criteria.
• Filtering: Include or exclude tests with certain tags during execution (useful in CI/CD pipelines).
• Organization: Organize tests into categories for better understanding and scope management.
• Reporting: Generate detailed reports highlighting specific areas of the software being tested.

Cypress Tags: Plugin Power to the Rescue

Cypress doesn't natively support tags. But don't worry, @bahmutov/cy-grep is a great Cypress plugin by Gleb Bahmutov to adds tag support.

Why Use @bahmutov/cy-grep?

• Frequent Updates: Gleb actively maintains the plugin, ensuring dependencies are up-to-date.
• Rapid Issue Resolution: Gleb quickly addresses reported issues.
• Community Recognition: Gleb is well-known and respected in the Cypress community.
• Real-time Grep: Ability to perform grep operations directly from the browser's DevTools console using the Cypress.grep() command when running your tests in 'open' mode.
• Failed tests Grep: Ability to run just the failed tests from the DevTools console using Cypress.grepFailed() command when running your tests in 'open' mode.

Installing @bahmutov/cy-grep: A Step-by-Step Guide

1. Install: npm i -D @bahmutov/cy-grep

2. Register in e2e.js:

   // cypress/support/e2e.js
   const registerCypressGrep = require('@bahmutov/cy-grep')
   registerCypressGrep()
   

   Copy

3. Load in cypress.config.js:

   // cypress.config.js
   const { defineConfig } = require("cypress");
   module.exports = defineConfig({
     e2e: {
       setupNodeEvents(on, config) {
         require('@bahmutov/cy-grep/src/plugin')(config);
         return config;
       },
     },
   });
   

   Copy

4. Configure Environment Variables: Set grepFilterSpecs and grepOmitFiltered to true in cypress.config.js.

   // cypress.config.js
   const { defineConfig } = require("cypress");
   module.exports = defineConfig({
     env: { grepFilterSpecs: true, grepOmitFiltered: true },
   });
   

   Copy

Using Tags in Cypress

Tags can be created at the group test level ( describe or context) and the test level ( it) by passing an object with a tags property as the function's second parameter. It is very common to use the character @ at the beginning of the tag name for clarity (like @smoke).

// test-tags.cy.js
describe('User Registration', { tags: ['@user-reg', '@smoke'] }, () => {
  it('should register a new user successfully', { tags: '@happy' }, () => {
    // Logic for testing successful user registration.
  });
  it('should not register with an already existing email', { tags: '@unhappy' }, () => {
    // Logic for testing registration with a duplicate email.
  });
});

Playwright Tags: Native Tagging for the Win

Playwright has built-in support for tags. A tag in Playwright must start with the character @. You set tags using the tag property in the details parameter when defining a test.

import { test, expect } from '@playwright/test';
test('Sample test with 1 tag', {
  tag: '@smoke',
}, async ({ page }) => {
  // ...
});
test('Sample test with 2 tags', {
  tag: ['@smoke', '@regression'],
}, async ({ page }) => {
  // ...
});

Tags also can be created at the group test level ( test.describe):

// test-tags.spec.ts
import { test, expect } from '@playwright/test';
test.describe('Sample group with tag', {
  tag: '@query',
}, () => {
  test('Sample test one', async ({ page }) => {
    // ...
  });
  test('Sample test two with 2 tags', {
    tag: ['@smoke', '@regression'],
  }, async ({ page }) => {
    // ...
  });
});

Test Filters: Precision Targeting in Cypress and Playwright

Filtering tests by tags, spec files, and test titles allows precise control over which tests to execute. This feature enhances efficiency by enabling QA Engineers to target specific tests relevant to their current work.

Cypress Test Filters:

Cypress can filter using:

• Spec files (natively supported).
• Test titles (using @bahmutov/cy-grep).
• Tags (using @bahmutov/cy-grep).

Filter by Spec Files

# Run a specific test file:
$ npx cypress run --spec "cypress/e2e/my-spec.cy.js"

# Run multiple test files:
$ npx cypress run --spec "cypress/e2e/examples/actions.cy.js,cypress/e2e/examples/files.cy.js"

# Run all tests in a folder:
$ npx cypress run --spec "cypress/e2e/login/**/*"

Filter by Test Title (with @bahmutov/cy-grep)

# Run all tests with "footer" in their title:
$ npx cypress run --env grep=footer

# Run all tests with "footer links" in their title:
$ npx cypress run --env grep="footer links"

# Run all tests WITHOUT "footer links" in their title:
$ npx cypress run --env grep="-footer links"

# Run tests with "footer", but without "links" in the titles:
$ npx cypress run --env grep="footer; -links"

Filter by Tags (with @bahmutov/cy-grep)

# Run suites and tests that have both tags @smoke and @regression:
$ npx cypress run --env grepTags="@smoke+@regression"

# Run suites and tests that have the tag @visual or the tag @regression:
$ npx cypress run --env grepTags="@visual @regression"

# Do not run any suites or tests that have the tag @quarantine:
$ npx cypress run --env grepTags=-@quarantine

# Run suites and tests with the tag @smoke but without the tag @quarantine:
$ npx cypress run --env grepTags=@smoke+-@quarantine

# Run suites or tests with the tag @visual or @regression but exclude those with the tag @quarantine:
$ npx cypress run --env grepTags = "@visual @regression --@quarantine"

Playwright Test Filters:

Playwright supports filtering by:

• Spec files (natively supported - implicit, by indicating the file).
• Test titles (natively supported with --grep).
• Tags (natively supported with --grep).

# Run tests with the tag @smoke:
playwright test --grep @smoke

# Run tests with the tag @smoke and @fast:
playwright test --grep "@smoke & @fast"

# Run tests with the tag @smoke or @fast:
playwright test --grep "@smoke | @fast"

# Exclude tests with the tag @slow:
playwright test --grep -@slow