Create End-to-end tests

End-to-end (E2E) testing ensures seamless system functionality and identifies errors before they reach production. This guide explores how to write end-to-end tests using Playwright. We'll cover everything from configuring Playwright to setting up Single Sign-On (SSO) and common pitfalls to avoid. By the end of this guide, you'll have a solid understanding of creating maintainable test suites that will help you catch errors before they become a problem.

What to consider when writing E2E tests

When writing E2E tests, you need to consider the following:

Install the app on a target environment. It ensures that you're testing the app in a similar environment to the user (for example, having SSO in place, not running the app on a development server, or similar).
Avoid hardcoded environment URLs and login information for a smooth login flow to an arbitrary environment. Instead, pass the necessary information as environment variables or read them from configuration files. It allows you to run your tests on different environments, even at various stages (such as development or hardening).

Install dependencies

To install the required dependencies, run the following command in your terminal:

npm init playwright@latest

Follow the prompt and select the following options:

Where to put your end-to-end tests?
- write e2e and press enter
Add a GitHub Actions workflow?
- select false and press enter
Install Playwright browsers (can be done manually via 'npx playwright install')?
- select true and press enter

Next, run the following command:

npm install --save-dev dotenv

The above command will install Playwright and the dotenv package. You'll need Playwright to run tests and the dotenv package to use .env files in your tests.

Configure Playwright

Ensure that you configure Playwright correctly so it can execute your end-to-end tests. Open the playwright.config.ts file in the root folder of your project and change the following content:

STORAGE_STATE_PATH: On the top of the file, add a new variable called STORAGE_STATE_PATH, which contains a path to a JSON file.

playwright.config.ts
// add those lines
import * as path from "path";
export const STORAGE_STATE_PATH = path.join(__dirname, 'playwright/.auth/credentials.json');
// existing config
export default defineConfig({...

projects: Playwright projects are logical groups of tests with the same configuration. For example, you can add a project for each browser. For the standard use case, specify a project for chromium and firefox.

playwright.config.ts
  /* Configure projects for major browsers */
  projects: [
    { name: 'setup', testMatch: /auth-setup\.ts/ },
    {
      name: 'chromium',
      use: {
        ...devices['Desktop Chrome'],
        storageState: STORAGE_STATE_PATH,
      },
      dependencies: ['setup'],
    },

    {
      name: 'firefox',
      use: {
        ...devices['Desktop Firefox'],
        storageState: STORAGE_STATE_PATH,
      },
      dependencies: ['setup'],
    },
  ],

use: The Playwright use property defines global options for all tests. Here is the basic configuration that you can use.

playwright.config.ts
use: {
    video: {
      mode: process.env.CI ? 'on': 'on-first-retry',
      size: { width: 800, height: 450 },
    },
    trace: {
      mode: process.env.CI ? 'retain-on-failure': 'on',
    },
    screenshot: {
      mode: process.env.CI ? 'on': 'only-on-failure',
      fullPage: false,
    },
    headless: !!process.env.CI,
    viewport: { width: 1920, height: 1080 },
  },

reporter: Playwright comes with a few built-in reporters for different needs, such as an HTML reporter

playwright.config.ts
reporter: process.env.CI
? [
    ['line'],
    ['junit', { outputFile: 'results.xml'}],
    ['html', { open: 'never'}],
  ]
: [['list'], ['html']],

Let's understand the configuration.

video: This property specifies if Playwright records a video while it executes the tests. This video is available in the HTML report. The properties defined will result in the following behavior:
- CI: Playwright always records a video.
- Local: Playwright records a video on the first test retry.
trace: This property specifies if and when Playwright records a trace when it executes the tests. The recorded trace is accessible in the HTML report. The properties defined will result in the following behavior:
- CI: The trace only persists if the test fails. Otherwise, Playwright will discard it.
- Local: The trace is always available.
screenshot: This property specifies if and when Playwright takes a screenshot at the end of the test or after a test fails. The screenshot is accessible in the HTML report. The properties defined will result in the following behavior:
- CI: The screenshot is always available.
- Local: The screenshot is only available when a test fails.
headless: This property specifies if the browser should run in headless or headed mode. The properties defined will result in the following behavior:
- CI: Playwright starts the browser in headless mode.
- Local: Playwright starts the browser in headed mode.
reporter: This property specifies which reporters Playwright should use. The properties defined will result in the following behavior:
- CI: Playwright will use the following reporters on the CI: line, junit, html
- Local: Playwright will use the following reporters locally: list, html

Set up single sign-on

Before running the end-to-end tests, you need to log in using your Dynatrace user credentials. Add the following content in the e2e/tests/auth/auth-setup.ts file.

e2e/src/utils/setup.ts: Create a new file called setup.ts within the e2e/src/utils directory. This file will contain the utilities necessary for the login process. Then, paste the following content into it:

e2e/src/utils/setup.ts
import { config } from 'dotenv';

// reads env vars from .env file
config();

interface Credentials {
  user: string;
  password: string;
}

// returns user name and password from env vars
export function getLoginCredentials(): Credentials {
  if (!!process.env.PLATFORM_INTEGRATION_TEST_USER && !!process.env.PLATFORM_INTEGRATION_TEST_PASSWORD) {
    return {
      user: process.env.PLATFORM_INTEGRATION_TEST_USER,
      password: process.env.PLATFORM_INTEGRATION_TEST_PASSWORD,
    };
  }
  throw Error(
    `Please create an .env file and set the variable PLATFORM_INTEGRATION_TEST_USER and PLATFORM_INTEGRATION_TEST_PASSWORD`,
  );
}

// returns url on which you want to execute the test
export const baseUrl = process.env.ENVIRONMENT_URL;

// returns the app id
export const appId = process.env.APP_ID_POSTFIX
  ? `my.dynatrace.notebooks.${process.env.APP_ID_POSTFIX.toLowerCase()}`
  : 'local-dev-server';

// extends app url with intent or local-dev-server if needed
export const generateSearch = (intent = '') => {
  const encodedIntent = intent ? `#${encodeURIComponent(intent)}` : '';
  const extraSearch = process.env.APP_ID_POSTFIX
    ? encodedIntent
    : `?locationAppIds=${encodeURIComponent('http://localhost:3000/ui,local-dev-server')}${encodedIntent}`;

  return extraSearch;
};

// returns the app path
export const appPathInShell = `/ui/apps/${appId}/${generateSearch()}`;

export const pageUrl = `${baseUrl}${appPathInShell}`;

e2e/tests/auth/auth-setup.ts: Create a new file e2e/tests/auth/auth-setup.ts, which will contain one test that gets the login credentials from e2e/src/utils/setup.ts and log you in. To achieve this, paste the following content into that file:

e2e/tests/auth/auth-setup.ts
import { test as setup, expect } from '@playwright/test';
import { STORAGE_STATE_PATH } from '../../../playwright.config';
import { getLoginCredentials, pageUrl } from '../../src/utils/setup';

setup('do login', async ({ page }) => {
  // go to app url
  await page.goto(pageUrl);
  // get user credentials
  const { user, password } = getLoginCredentials();

  /// login
  await page.getByLabel('Email').fill(user);
  await page.getByRole('button', { name: 'Next' }).click();
  await page.getByRole('textbox', { name: 'Password' }).fill(password);
  await page.getByRole('button', { name: 'Sign in' }).click();

  // wait until platform is loaded
  await expect(page.getByTestId('app-iframe')).toBeAttached({ timeout: 10_000 });

  // Persist the logged in state
  await page.context().storageState({ path: STORAGE_STATE_PATH });
});

Let's understand the code.

e2e/src/utils/setup.ts:
- The config import is from the dotenv package, which loads environment variables from a .env file.
- The Credentials interface defines the structure of login credentials, with user and password properties.
- The getLoginCredentials() function retrieves the login credentials from environment variables (PLATFORM_INTEGRATION_TEST_USER and PLATFORM_INTEGRATION_TEST_PASSWORD). If you don't set the variables, the function will throw an error, requesting you to set the necessary environment variables.
- The code snippet stores the environment variables ENVIRONMENT_URL and APP_ID in baseUrl and appId, respectively, and constructs pageUrl by combining environmentUrl and appId using template literals.
e2e/tests/auth/auth-setup.ts:
- The test as setup import from @playwright/test means that this test is only used for setup purposes and won't test an actual feature. The STORAGE_STATE is a path specified in the playwright.config.ts file that defines where Playwright should store the authentication state. The authentication setup uses the getLoginCredentials, pageUrl import from ../app/setup to access the login credentials and URL that points directly to the app.
- The await page.context().storageState({ path: STORAGE_STATE }); defines where Playwright should store the login state. Playwright will use this login state for all other tests to avoid having to authenticate for each test.

Write your first test

To write a clean first test, you'll need three layers of files, page objects, assertions and the test file itself.

e2e/src/page-objects/app-landing-page/app-header.po.ts: First, let's create a page-object containing selectors and methods to interact with your app. In this case, this page object targets the app header. Now, create a new file named e2e/src/page-objects/app-landing-page/app-header.po.ts and paste the following content into it.

e2e/src/page-objects/app-landing-page/app-header.po.ts
import { FrameLocator, Locator, Page } from '@playwright/test';

export class AppHeader {
  private readonly APP_IFRAME: FrameLocator;
  public readonly HEADER: Locator;

  constructor(private page: Page) {
    this.APP_IFRAME = page.frameLocator('[data-testid="app-iframe"]');
    this.HEADER = this.APP_IFRAME.getByRole('navigation');
  }

  async clickAppIcon() {
    await this.HEADER.click();
  }
}

e2e/src/page-objects/app-landing-page/app-header.assertion.ts: This assertion file only contains methods for asserting the page state of your app. In this case, the assertion targets the app header. Next, create a new file named e2e/src/page-objects/app-landing-page/app-header.assertion.ts and paste the following content into it.
Note
Moving assertions into a separate file is only helpful in two cases:
- You're using them in several tests or intend to do so.
- They're complex and long. In this case, extract them in an assertion file and give them a descriptive name.
e2e/src/page-objects/app-landing-page/app-header.assertion.ts
```
import { Page, expect } from '@playwright/test';
import { AppHeader } from './app-header.po';

export class VerifyAppHeader {
  public readonly AppHeader: AppHeader;

  constructor(private page: Page) {
    this.AppHeader = new AppHeader(page);
  }

  async verifyAppIsOpen() {
    await expect(this.AppHeader.HEADER).toBeVisible();
  }
}
```

e2e/tests/landing-page/first-test.spec.ts: Now that you've configured everything, let's write the first test. Create a test file first-test.spec.ts in your project's e2e/tests/landing-page/ directory and add the following content.

e2e/tests/landing-page/first-test.spec.ts
import { test } from '@playwright/test';
import { VerifyAppHeader } from '../../src/page-objects/app-landing-page/app-header.assertion';
import { pageUrl } from '../../src/utils/setup';

test.beforeEach(async ({ page }) => {
  await page.goto(pageUrl);
});

test('Should open the app successfully', async ({ page }) => {
  await new VerifyAppHeader(page).verifyAppIsOpen();
});

Let's understand the files.

Page objects: A page object represents your UI in test code. This file includes selectors and methods to interact with your page while testing.
- Selectors are usually defined on top of a page object and initialized in the constructor. Playwright recommends using their built-in locators instead of Xpath or CSS selectors. Additionally, selector names should follow the SCREAMING_SNAKE_CASE format.
- Interaction methods execute a logically related group of actions on a specific page. The method's name should always target the functionality or the outcome, such as openApp or executeQuery.
Assertion: An assertion file should generally relate to a page object. For the page object app-header.po.ts, the connected assertion file would be app-header.assertion.ts. Additionally, the class name should be the same as the one of the page object but prefixed with Verify.
Test file: The test function defines an individual test case and in this case. We've named the test Should open the app successfully. Within the test, we use Playwright methods on page (the Page) to interact with the page and perform assertions. The test does two things:
- await page.goto(pageUrl): The beforeEach method calls this function, which means it runs before every test in that file. The page.goto method redirects the test to a new URL, which is in this case the pageUrl coming from the e2e/src/utils/setup.ts file.
- await new VerifyAppHeader(page).verifyAppIsOpen(): This method calls the verifyAppIsOpen method from the'e2e/src/page-objects/app-landing-page/app-header.assertion.ts` file we created earlier which asserts that the header div is visible.

Run tests

To run your tests, you'll need to do the following:

Set the environment.
Add npm scripts.
Ignore Playwright output folders to avoid app reloads.
Run the tests locally or in your CI.

Here's how:

Set environment

Ensure that you configure environment variables before running the tests. The setup supports the .env file if you run tests locally. Create a .env file in the root directory and add the following content:

.env
ENVIRONMENT_URL=<Your-Environment-Url>
APP_ID=<Your-App-Id>
PLATFORM_INTEGRATION_TEST_USER=<Your-Test-User-Email>
PLATFORM_INTEGRATION_TEST_PASSWORD=<Your-Test-User-Password>

Note

Replace the following placeholder:

<Your-Environment-Url> with the URL of your Dynatrace environment that hosts your app. Ensure it doesn't end with a /, which would cause failures while running the tests.
<Your-App-Id> with your app's ID.
<Your-Test-User-Email> with the email address of your test user.
<Your-Test-User-Password> with the password of your test user.

Add `npm` scripts

To run the tests, add the following npm scripts in your package.json file:

package.json
{
  "scripts": {
    "test:e2e": "playwright test --project=chromium",
    "test:e2e-ci": "playwright test"
  }
}

Ignore Playwright output folders to avoid app reloads

Playwright produces output files when it executes tests. The app's hot reloading detects these files by default and triggers page reloads. Configure the hot reloading mechanism to ignore Playwright output files to avoid such reloads.

app.config.json
{
  ...
  "dev": {
    "fileWatcher": {
      "ignore": ["test-results/**", "playwright/**", "playwright-report/**"]
    }
  }
}

Run tests locally

To run the tests locally, execute the following command in the terminal:

npm run test:e2e

Run tests in CI

To run your tests in CI, follow these steps:

Install the app: Install the new version of your app on the desired environment. The main thing to remember is that you don't override an existing app. Choose a unique app ID only to test your app using environment variables in app.config.ts.
Run the tests: Since you have configured headless mode in the config, just run the following command in your CI: npm run test:e2e-ci. Ensure that you've configured the required environment variables.
Uninstall the app: You should always uninstall the app, regardless of whether tests fail or pass.

Debug tests

Writing tests is similar to writing code. You might face a situation where you want to debug your tests. In Playwright, there are some easy ways to debug your tests. Check out the Playwright documentation to learn more about its helpful debugging possibilities.

Still have questions?

Find answers in the Dynatrace Community

Go to Dynatrace Community

What to consider when writing E2E tests​

Install dependencies​

Configure Playwright​

Set up single sign-on​

Write your first test​

Run tests​

Set environment​

Add npm scripts​

Ignore Playwright output folders to avoid app reloads​

Run tests locally​

Run tests in CI​

Debug tests​