Cypress vs. Playwright: Mastering Test Execution with Tags and Filters

This article explores how to control your Cypress and Playwright tests using tags and filters, including practical examples and plugin recommendations. This builds upon our previous exploration of annotations and grouping.

Take Control of Your Tests: Tags and Filters

Cypress and Playwright offer powerful ways to manage and execute your tests. Let's explore tags and filters in both frameworks to gain complete control.

Why Use Tags for Test Management?

Tags are crucial for efficient test organization. They help categorize tests based on type, priority, or feature, making it easier to manage and execute specific subsets of tests.

Tags enable:

• Test Selection: Run specific tests (e.g., "smoke," "regression") based on criteria.
• Filtering: Include or exclude tests during execution, which is especially useful in CI/CD pipelines.
• Organization: Group tests logically for better understanding.
• Reporting: Generate detailed reports that highlight coverage and identify gaps.

Let's explore how to implement and use tags effectively in both Cypress and Playwright.

Cypress Tags: Extending Functionality with Plugins

Cypress doesn't natively support tags. But don't worry! Plugins come to the rescue. Here are two popular options:

• @cypress/grep: The official Cypress plugin for tag support.
• @bahmutov/cy-grep: Gleb Bahmutov's enhanced alternative.

Why Choose @bahmutov/cy-grep?

While @cypress/grep is widely used, @bahmutov/cy-grep offers several advantages:

• Frequent Updates: Gleb actively maintains the plugin.
• Quick Issue Resolution: Reported problems are addressed rapidly.
• Community Recognition: Gleb is a recognized Cypress expert.
• DevTools Integration: Perform grep operations directly from the browser console in "open" mode using Cypress.grep(). You can also run failed tests with Cypress.grepFailed().

Let's focus on @bahmutov/cy-grep for the rest of this article due to its enhanced features and active maintenance. It offers filtering Cypress tests using tags.

Installing and Configuring @bahmutov/cy-grep

1. Install: Add the plugin as a dev dependency: npm i -D @bahmutov/cy-grep

2. Register: Load the module in cypress/support/e2e.js:

   const registerCypressGrep = require('@bahmutov/cy-grep')
   registerCypressGrep()
   

   Copy

3. Configure: Load and register the module in cypress.config.js:

   const { defineConfig } = require("cypress");
   module.exports = defineConfig({
     e2e: {
       setupNodeEvents(on, config) {
         require('@bahmutov/cy-grep/src/plugin')(config);
         return config;
       },
     },
   });
   

   Copy

4. Set Environment Variables: Configure grepFilterSpecs and grepOmitFiltered to true in cypress.config.js:

   module.exports = defineConfig({
       env: {
           grepFilterSpecs: true,
           grepOmitFiltered: true
       },
   });
   

   Copy

Tagging Tests in Cypress

Use the tags property within the describe or it blocks to assign tags:

describe('User Registration', { tags: ['@user-reg', '@smoke'] }, () => {
  it('should register a new user successfully', { tags: '@happy' }, () => {
    // Test logic
  });
});

Now you can use these tags to filter tests during execution!

Playwright Tags: Native and Powerful

Playwright natively supports tags, offering an easy and efficient way to categorize your tests.

How to Add Tags in Playwright

Use the tag property in the test or test.describe blocks:

import { test, expect } from '@playwright/test';

test('Sample test with 1 tag', {
  tag: '@smoke',
}, async ({ page }) => {
  // Test logic
});

test.describe('Sample group with tag', {
  tag: '@query',
}, () => {
  test('Sample test one', async ({ page }) => {
    // Test logic
  });
});

Test Filters: Precision in Execution

Filtering tests by tags, spec files, and test titles lets you target specific tests, increasing efficiency and focusing testing cycles.

Cypress Test Filters: Spec Files, Titles, and Tags

Cypress allows filtering by spec files natively. Using plugins extends this to tags and test titles.

Filter by Spec Files

Use the --spec parameter:

• Specific file: $ npx cypress run --spec "cypress/e2e/my-spec.cy.js"
• Multiple files: $ npx cypress run --spec "cypress/e2e/examples/actions.cy.js,cypress/e2e/examples/files.cy.js"
• Glob pattern: $ npx cypress run --spec "cypress/e2e/login/**/*"

Filter by Test Title (with @bahmutov/cy-grep)

Use the --env grep=... option:

• Title substring: $ npx cypress run --env grep="footer links"
• Multiple substrings: $ npx cypress run --env grep="footer links; header links"
• Invert filter: $ npx cypress run --env grep="-footer links"

Filter by Tags (with @bahmutov/cy-grep)

Use the --env grepTags=... option:

• Run tests with both tags @smoke and @regression: $ npx cypress run --env grepTags="@smoke+@regression"
• Run tests with either @visual or @regression: $ npx cypress run --env grepTags="@visual @regression"
• Exclude tests with the @quarantine tag: $ npx cypress run --env grepTags="-@quarantine"
• Exclude tests with @quarantine, even if they have another tag: $ npx cypress run --env grepTags="@visual @regression --@quarantine"

Playwright Test Filters: Native Tag Filtering

Playwright provides native support for filtering tests by tags using the --grep option:

• Run tests with the @smoke tag: playwright test --grep @smoke
• Run tests with either @smoke or @regression: playwright test --grep "@smoke|@regression"
• Run tests that don't have the @slow tag: playwright test --grep /^(?!.*@slow).*$/

Conclusion: Mastering Test Control

By mastering tags and test execution filters in Cypress and Playwright, you'll improve testing efficiency. You can run focused tests, streamline CI/CD pipelines, and maintain a well-organized test suite.