Skip to main content

Test custom actions

After you've created your custom Workflow Action, you'll likely want to test it. This guide describes best practices and recommended approaches to properly test your app functions and widgets that implement the custom action.

Approach custom action tests​

Run app functions unit and integration tests in your terminal or IDE with mocked dependencies. For end-to-end tests, use AutomationEngine via the Workflows API.

For widgets, follow Dynatrace Apps testing guide for unit tests and use the dev-helper for end-to-end tests.

Test an app function​

Refer to the Dynatrace Apps testing guide on how to write tests for app functions.

Write unit and integration tests​

The app function testing guide emphasizes testing the business logic of functions. We recommend focusing on testing individual function invocations and simulating dependencies and requests. This approach allows isolated testing of core functionality to ensure proper behavior. By simplifying the testing environment and abstracting away external dependencies, you can effectively evaluate if your app functions work as expected.

Write E2E tests​

Note

Your test workflow will use the actions currently installed in the target environment. During the setup, deploy and ensure that the action version you expect to test is correct. If your custom action uses the settings schema for connection details, also ensure that:

  • You've created settings schemas.
  • Connections objects exist.

The E2E testing guide emphasizes testing the interaction between your application and third-party systems. Verifying seamless integration with these systems is essential to avoid regressions.

We recommend creating your test scenarios as workflows using a Workflows API. Run these workflows and inspect the results of their execution. The testing process shouldn't involve any user interface interactions.

By utilizing the Workflows API, you can simulate various scenarios and test the interactions and behavior of your app function with Dynatrace or third party systems. Analyzing the workflow execution results will help you identify any issues or regressions of your app functions.

What you should test:

  • Single invocations with different parameters for test scenarios.
  • Several app functions tested together; those are scenarios where the input of the subsequent app function invocations depends on the results of the previous one.
  • E2E tests are the only place where you should use Jinja expressions. All other tests should avoid the syntax. AutomationEngine resolves them to static values, which your actions later use as inputs.
Note

In some test cases, you may expect an app function that implements an action to fail intentionally, such as receiving an HTTP 403 response from a request. In such situations, we recommend creating an intermediate placeholder task with a custom start condition to validate if the previous task failed as expected.

This approach allows you to verify the desired behavior explicitly and ensures that the initial tasks, based on their expected outcome, trigger subsequent tasks. By incorporating these intermediate checks, you can effectively test and validate how your app handles failure scenarios.

Example​

The example below shows a TypeScript script that leverages the Automation SDK to create and run a workflow. The script also checks for the final state of the workflow execution.

The workflow consists of three tasks in a sequence:

  • Make a Jira issue.
  • Comment on the Jira issue.
  • Transfer the issue to a different state.
Note

Using the Workflow API, you can use any programming language and tools for such tests. To access its visual documentation, navigate to https://<Your-Environment-Url>/platform/swagger-ui/index.html?urls.primaryName=Automation

Remember to replace <Your-Environmental-Url> with your environmental URL address in the link above.

action-e2e.spec.ts
import { workflowsClient, ExecutionState, executionsClient } from '@dynatrace-sdk/client-automation';

const workflow = await workflowsClient.createWorkflow({
  body: {
    title: '[e2e] Test Jira actions',
    tasks: {
      create_issue: {
        name: 'create_issue',
        input: {
          project: {
            id: '18890',
          },
          issueType: {
            id: '2',
          },
          description: 'New issue',
          connectionId:
            'vu9U3hXa3q0AAAABAB1hcHA6ZHluYXRyYWNlLmppcmE6Y29ubmVjdGlvbgAGdGVuYW50AAZ0ZW5hbnQAJGI4MjU2NDI1LTM5YWEtM2ZlMi05NzgzLTlkOGE1ZmQ0MWFkY77vVN4V2t6t',
        },
        action: 'dynatrace.jira:jira-create-issue',
        position: {
          x: 0,
          y: 1,
        },
      },
      comment_on_issue: {
        name: 'comment_on_issue',
        input: {
          comment: 'Updating with relevant details',
          issueID: '{{ result("create_issue").id }}',
          connectionId:
            'vu9U3hXa3q0AAAABAB1hcHA6ZHluYXRyYWNlLmppcmE6Y29ubmVjdGlvbgAGdGVuYW50AAZ0ZW5hbnQAJGI4MjU2NDI1LTM5YWEtM2ZlMi05NzgzLTlkOGE1ZmQ0MWFkY77vVN4V2t6t',
        },
        action: 'dynatrace.jira:jira-add-comment',
        position: {
          x: 0,
          y: 2,
        },
        conditions: {
          states: {
            create_issue: 'OK',
          },
        },
        description: 'Comment on a Jira issue',
        predecessors: ['create_issue'],
      },
      transition_issue: {
        name: 'transition_issue',
        input: {
          project: '18890',
          issue: '{{ result("create_issue") }}',
          issueType: '2',
          connectionId:
            'vu9U3hXa3q0AAAABAB1hcHA6ZHluYXRyYWNlLmppcmE6Y29ubmVjdGlvbgAGdGVuYW50AAZ0ZW5hbnQAJGI4MjU2NDI1LTM5YWEtM2ZlMi05NzgzLTlkOGE1ZmQ0MWFkY77vVN4V2t6t',
          targetStatus: '11415',
        },
        action: 'dynatrace.jira:jira-transition-issue',
        position: {
          x: 0,
          y: 3,
        },
        conditions: {
          states: {
            comment_on_issue: 'OK',
          },
        },
        predecessors: ['comment_on_issue'],
      },
    },
  },
});

// Utility function to block the script execution for `ms` milliseconds.
const sleep = (ms: number) => new Promise((resolve) => setTimeout(resolve, ms));
const POLL_INTERVAL = 5 * 1000; // 5 seconds

let execution = await workflowsClient.runWorkflow({ id: workflow.id as string, body: {} });
// Poll the execution state until it's finished.
while (execution.state === ExecutionState.Idle || execution.state === ExecutionState.Running) {
  execution = await executionsClient.getExecution({ id: execution.id as string });
  await sleep(POLL_INTERVAL);
}

if (execution.state !== ExecutionState.Success) {
  console.error('E2E tests failed');
  process.exit(1);
}

Test a widget​

You don't need to test any Strato design system components. You can rely on them; developers have tested them adequately.

Write UI integration tests​

Visit the Dynatrace Apps testing guide to learn how to write unit tests for your UI. Focus on testing the functionality of a single component in isolation by simulating how your end users will use it.

Write E2E tests​

Test your widget only by performing actions you could expect real users to make. The test framework allows initializing a widget with or without the initial value, such as an existing or new task configuration. More details are in the example below.

You don't need to worry about embedding your widget into the Workflows or how it persists the widget value. Workflows take care of this for you.

The tests you write must add value to your widget, verify the widget's state, and show that:

  • Your components interact as expected; for example, there are interdependencies between widget UI components.
  • Your widget compiles and renders all fields. Ensure that there are no breaking changes in dependencies.
  • The widget produces the output you would expect.

It's important not to test the app function or its business logic. Therefore, don't use Run Action from the dev-helper view. You can test app functions using specific app function tests. In these E2E tests, we focus on widgets only.

Examples​

These examples test the Greeter widget created in the custom Workflows action section.

Let's begin by setting up E2E tests for Dynatrace Apps. In addition to the required environment variables, add WIDGET_URL, which refers to the URL address where your custom action runs. It could either be the action's local URL displayed by the Dynatrace App Toolkit or the URL of the deployed widget.

Note

You can retrieve the base URL of a deployed action using the visual documentation by navigating to: https://<Your-Environment-Url>/platform/swagger-ui/index.html?urls.primaryName=AppEngine%20-%20Registry#/AppEngine%20-%20Registry%20-%20Apps/getApp

Provide your application ID on this page, add isolatedUri to the add-fields parameter, and execute the request. In the response, look for the isolatedUri object. Under this object, you'll find the widgetUrl. Finally, append your action widget's specific path, for example, /actions/greeter, to the widgetUrl while making sure you preserve any query parameters that may be present in the widgetUrl e.g. https://<widgetUrl>/actions/greeter?<Your-Query-Params>.

Ensure you replace the <Your-Environment-Url> placeholder with the actual URL address of your environment.

Once you've configured the tests, you can change the setup method in the setup.ts file. This change will allow the setup method to accept a URL parameter. When you provide a URL address, the test will navigate to this location just before it starts. Providing a URL eliminates the need to switch between iframes during the test.

e2e/setup.ts
export async function setup(
  controller: TestController,
  navigateTo?: string
): Promise<void> {
  await controller.maximizeWindow();
  await controller.useRole(role);
  if (navigateTo) {
    await controller.navigateTo(navigateTo);
 }

 ...
  const iframeSelector = Selector("iframe", { timeout: 60_000 });
  await controller.switchToIframe(iframeSelector);
}

Additionally, add the following content:

e2e/setup.ts
export const DEV_HELPER_URL = `${process.env.ENVIRONMENT_URL}/ui/apps/dynatrace.automations/dev-helper?src=${process.env.WIDGET_URL}`;
export const WIDGET_STATE_STORAGE_KEY = 'dev-helper-widget-state';
  • The DEV_HELPER_URL represents the URL address to the dev-helper, which includes a query parameter src pointing to the widget URL. Refer to the debugging custom action documentation for more detailed information on accessing dev-helper.
  • The WIDGET_STATE_STORAGE_KEY is the key used in the localStorage of the dev-helper to store the widget state.
Note

Be aware that your custom action may not be installed in an environment where the E2E tests run. This circumstance could lead to an error page indicating that the provided app doesn't exist. However, the test setup will redirect you to the dev-helper within a few seconds.

Test an empty widget​

Let's create the initial test that guarantees the persistence of provided values in the widget state.

e2e/tests/greeter-empty.spec.ts
import { ClientFunction, Selector } from 'testcafe';
import { DEV_HELPER_URL, WIDGET_STATE_STORAGE_KEY, setup } from '../setup';

// Client function to run custom-code on the client side.
// https://testcafe.io/documentation/402832/guides/basic-guides/client-functions
const getLocalStorageItem = ClientFunction((key: string) => window.localStorage.getItem(key));

fixture('Empty widget').beforeEach(async (t) => {
  await setup(t, DEV_HELPER_URL);
  await t.switchToIframe(Selector('iframe#widget', { timeout: 10_000 }));
});

test('Fill empty widget form', async (t) => {
  const nameInput = Selector('[data-testid="base-code-editor-content"]');

  // Fill widgets inputs
  await t.typeText(nameInput, 'John Sr.');

  // Assert inputs values
  await t.expect(nameInput.innerText).eql('John Sr.');

  // Switch to dev-helper app iframe
  await t.switchToMainWindow().switchToIframe(Selector('iframe'));
  // Assert widget state
  await t.expect(getLocalStorageItem(WIDGET_STATE_STORAGE_KEY)).eql(JSON.stringify({ name: 'John Sr.' }));
});

Let's review the most crucial parts of the test:

  • We set up the test and navigate to the dev-helper. Then, we switch to the context of the widget iframe.
    Note

    To make it convenient, you can always find the iframe of any widget within the dev-helper by the ID widget.

  • We provide value to the widget input.
  • We switch to the dev-helper iframe, retrieve the widget state, and verify that it matches the expected value, which acts as a contract. The AutomationEngine will use it during the execution of the action upon saving a workflow.
Test a widget with an initial value​

The following test shows how to pass the initial value via the URL query parameters.

e2e/tests/greeter-with-initial-value.spec.ts
import { ClientFunction, Selector } from 'testcafe';
import { DEV_HELPER_URL, WIDGET_STATE_STORAGE_KEY, setup } from '../setup';

// Client function to run custom-code on the client side.
// https://testcafe.io/documentation/402832/guides/basic-guides/client-functions
const getLocalStorageItem = ClientFunction((key: string) => window.localStorage.getItem(key));

fixture('Widget with initial value').beforeEach(async (t) => {
  const initialValue = { name: 'Mark' };
  const uriEncodedInitialValue = encodeURIComponent(JSON.stringify(initialValue));
  await setup(t, `${DEV_HELPER_URL}&initialValue=${uriEncodedInitialValue}`);
  await t.switchToIframe(Selector('iframe#widget', { timeout: 10_000 }));
});

test('Validate initial values and make changes', async (t) => {
  const nameInput = Selector('[data-testid="base-code-editor-content"]');

  // Verify that the widget holds the correct value
  await t.expect(nameInput.innerText).eql('Mark');

  // Make changes
  await t.typeText(nameInput, 'Ann Marie', { replace: true });

  // Switch to dev-helper app iframe
  await t.switchToMainWindow().switchToIframe(Selector('iframe'));
  // Assert widget state
  await t.expect(getLocalStorageItem(WIDGET_STATE_STORAGE_KEY)).eql(JSON.stringify({ name: 'Ann Marie' }));
});

Let's understand what's happening in the test:

  • We set up the test and navigated to the dev-helper. This time, we include an extra query parameter called initialValue. It should hold the URI-encoded value of the initial value for your widget. The dev-helper passes this value to the widget during its initial rendering. Finally, switch to the context of the widget iframe.
  • We ensure the widget holds the correct initial value.
  • After making changes, the remaining part of the test ensures that the widget state aligns with our expectations.

Add optional test improvements​

Avoid repetitions for more extensive test scenarios while looking for page elements or simulating user actions. To do so, we'll create Page Models. For further details, refer to the testcafe documentation.

Create a file, e2e/page-models/dev-helper.ts, with the following content.

e2e/page-models/dev-helper.ts
import { Selector, t } from 'testcafe';

class DevHelper {
  widgetIframe: Selector;

  constructor() {
    this.widgetIframe = Selector('iframe#widget', { timeout: 10_000 });
  }

  async switchToDevHelper() {
    await t.switchToMainWindow().switchToIframe(Selector('iframe'));
  }

  async switchToWidgetIframe() {
    await t.switchToIframe(this.widgetIframe);
  }
}

export default new DevHelper();
  • The widgetIframe selector targets the widget iframe.
  • The switchToDevHelper action provides a convenient method to switch to the dev-helper iframe. Such action is practical, for example, when you need to access and read the widget state from the localStorage.
  • The switchToWidgetIframe action offers a convenient way to switch to the widget iframe and interact with it.

As a next step, create a Greeter page model in e2e/page-models/greeter.ts.

e2e/page-models/greeter.ts
import { Selector, t } from 'testcafe';

class Greeter {
  nameInput: Selector;

  constructor() {
    this.nameInput = Selector('[data-testid="base-code-editor-content"]');
  }

  async typeName(name: string) {
    await t
      .expect(this.nameInput.exists)
      .ok('name input is not visible')
      .typeText(this.nameInput, name, { replace: true });
  }
}

export default new Greeter();

To ease the step of getting widget state, you can create the following file e2e/utils.ts and add content:

e2e/utils.ts
import { ClientFunction } from 'testcafe';
import DevHelper from './page-models/dev-helper';
import { WIDGET_STATE_STORAGE_KEY } from './setup';

export const getLocalStorageItem = ClientFunction((key: string) => window.localStorage.getItem(key));

export const getWidgetState = async (): Promise<Record<string, unknown>> => {
  await DevHelper.switchToDevHelper();
  const widgetStateRaw: string | null = await getLocalStorageItem(WIDGET_STATE_STORAGE_KEY);
  if (!widgetStateRaw) return {};
  try {
    return JSON.parse(widgetStateRaw);
  } catch (e) {
    console.warn('Failed to read widget state due to', e);
    return {};
  }
};

export const encodeInitialValue = (value: Record<string, unknown>): string => encodeURIComponent(JSON.stringify(value));

The getWidgetState gets the widget state as an object from the localStorage, and the `encodeInitialValue "utility function helps to translate the object value to URI encoded string.

Below, you can find updated tests that use Page Models objects and actions.

Test an empty widget​

e2e/tests/greeter-empty.spec.ts
import DevHelper from '../page-models/dev-helper';
import Greeter from '../page-models/greeter';
import { setup, DEV_HELPER_URL } from '../setup';
import { getWidgetState } from '../utils';

fixture('Empty widget').beforeEach(async (t) => {
  await setup(t, DEV_HELPER_URL);
  await DevHelper.switchToWidgetIframe();
});

test('Fill empty widget form', async (t) => {
  // fill widget input
  await Greeter.typeName('John Sr.');

  // assert inputs values
  await t.expect(Greeter.nameInput.innerText).eql('John Sr.');

  // assert widget state
  await t.expect(await getWidgetState()).eql({ name: 'John Sr.' });
});

Test a widget with an initial value​

e2e/tests/greeter-with-initial-value.spec.ts
import DevHelper from '../page-models/dev-helper';
import Greeter from '../page-models/greeter';
import { DEV_HELPER_URL, setup } from '../setup';
import { encodeInitialValue, getWidgetState } from '../utils';

fixture('Widget with initial value').beforeEach(async (t) => {
  const initialValue = { name: 'Mark' };
  await setup(t, `${DEV_HELPER_URL}&initialValue=${encodeInitialValue(initialValue)}`);
  await DevHelper.switchToWidgetIframe();
});

test('Validate initial values and make changes', async (t) => {
  // verify that the widget value has been populated correctly
  await t.expect(Greeter.nameInput.innerText).eql('Mark');

  // make changes
  await Greeter.typeName('Ann Marie');

  // assert widget state
  await t.expect(await getWidgetState()).eql({ name: 'Ann Marie' });
});
Note

When using TestCafe with a headless browser like Chrome Headless to test your custom actions, you may get the following error:

JS ERROR | Cannot read properties of null (reading 'hammerhead|shadow-ui-element')

Disable TestCafe's native automation mode to solve this issue by following the instructions on TestCafe's website

Still have questions?
Find answers in the Dynatrace Community