Store static data in Grail

• How-to
• 5 minutes

The lookup upload API allows you to upload static data and store it as a tabular file in the . This guide will walk you through using the /files/tabular/lookup:upload endpoint to upload your data.

Use case

Imagine a scenario where you're storing user event information in Grail. This data includes the user's email, current balance, and the currency of the balance. You want to convert the balance into a different currency using conversion rates fetched from a third-party API. By storing the user event data and the conversion rates in Grail, you can perform this conversion and generate balances in the desired currency directly within Grail.

User event data structure

Here is an example of the user event data stored in Grail:

[
  {
    "email": "first@first.com",
    "balance": 10,
    "currency": "USD"
  },
  {
    "email": "second@first.com",
    "balance": 130,
    "currency": "USD"
  },
  {
    "email": "third@first.com",
    "balance": 20,
    "currency": "USD"
  }
]

Conversion rate data

The conversion rates, fetched from a third-party API, are stored in Grail as follows:

[
  { "source": "USD", "EUR": 0.88, "AUD": 1.39, "PLN": 3.94 },
  { "source": "EUR", "USD": 1.14, "AUD": 0.72, "PLN": 0.25 }
]

You can calculate each user's balance in a specific currency using these two datasets. For example, you can convert USD balances into EUR using the stored conversion rates.

Upload static data in Grail

The endpoint is used to upload lookup data and store it as a new tabular file or replace an existing one. The data is parsed using the Dynatrace Pattern Language (DPL), which allows you to define patterns for extracting records from the uploaded data.

The request must be submitted as multipart/form-data with the following parts:

• request: Contains the request parameters in JSON format
• content: Contains the lookup data in text format

Key request parameters

Here are details of important request parameters:

• filePath (required): A file path where the data will be stored in Grail. It must start with a /lookups/ prefix. It is later used to fetch the data.
• parsePattern (required): A DPL (Dynatrace Pattern Language) expression to parse the uploaded data.
• lookupField (required): The unique field in the lookup data that acts as ID.
• overwrite: If set to true, the API will overwrite previously written content in the file.
• displayName: A user-friendly name for the file.
• description: A description of the file.
• skippedRecords: The number of initial records to skip in the uploaded data. The default is 0.
• timezone: The timezone for parsing time and date fields.
• locale: The local for parsing locale-specific information.
• autoFlatten: Set it to true to extract nested fields to the root level when the specified DPL pattern results in a single record-type field.

Upload data in Dynatrace app

In the following example, when the user selects the button Store Currency rates, the app will upload the currency rate information to the Grail Resource Store into filePath /lookups/currencyrates. In the example, we're hardcoding the currency rates, but you can easily use an app function to fetch the data from a third-party API and store it.

ui/app/pages/Home.tsx

import React, { useCallback } from 'react';
import { Flex } from '@dynatrace/strato-components/layouts';
import { Button } from '@dynatrace/strato-components/buttons';

const data = [
  { source: 'USD', EUR: 0.88, AUD: 1.39, PLN: 3.94 },
  { source: 'EUR', USD: 1.14, AUD: 0.72, PLN: 0.25 },
];

export const Home = () => {
  const handleClick = useCallback(() => {
    if (!data) {
      console.error('No conversion rate data available.');
      return;
    }

    const request = {
      lookupField: 'data',
      filePath: '/lookups/currencyrates',
      overwrite: true,
      displayName: 'Currency Rates',
      skippedRecords: 0,
      autoFlatten: false,
      timezone: 'UTC',
      locale: 'en_US',
      description: 'Currency Rates for Conversion',
      parsePattern: 'JSON:data',
    };

    const blob = new Blob([JSON.stringify(data)], {
      type: 'application/json',
    });
    const file = new File([blob], 'currencyrates.json', {
      type: 'application/json',
    });

    const formData = new FormData();
    formData.append('request', JSON.stringify(request));
    formData.append('content', file);

    fetch('/platform/storage/resource-store/v1/files/tabular/lookup:upload', {
      method: 'POST',
      body: formData,
    })
      .then((response) => {
        if (!response.ok) {
          throw new Error(`HTTP error! status: ${response.status}`);
        }
      })
      .then((json) => {
        console.log('Lookup data uploaded successfully:');
      })
      .catch((error) => {
        console.error('Error uploading lookup data:', error);
      });
  }, []);

  return (
    <Flex flexDirection="column" alignItems="center" padding={32}>
      <Button onClick={handleClick}>Store Currency Rates</Button>
    </Flex>
  );
};

Tip

This operation requires the scope storage:files:write. Read more about scopes in this guide.

Fetch the uploaded data

The uploaded data in Grail can be accessed using the DQL load command. Here is a DQL example:

load "/lookups/currencyrates"

Fetch data in Dynatrace app

In the following example, we're mimicking the real user event data with DQL's data command and converting the balance in USD into EUR with the uploaded currency rates. We'll fetch this data using the useDql React Hook and show it in the Strato DataTableV2 component.

ui/app/App.tsx

import React from 'react';
import { Flex } from '@dynatrace/strato-components/layouts';
import { useDql } from '@dynatrace-sdk/react-hooks';
import { DataTableV2 } from '@dynatrace/strato-components-preview/tables';
import { convertToColumnsV2 } from '@dynatrace/strato-components-preview/conversion-utilities';

export const App = () => {
  const query = `data json:"""
  [
    {
        "email": "first@first.com",
        "balance": 10,
        "currency": "USD"
    },
    {
        "email": "second@first.com",
        "balance": 130,
        "currency": "USD"
    },
    {
        "email": "third@first.com",
        "balance": 20,
        "currency": "USD"
    }
  ]
  """
| fieldsAdd eurBalance = balance * lookup([
  load "/lookups/currencyrates"], sourceField: currency, lookupField:data[source])[data][EUR]`;

  const result = useDql(query);

  return (
    <Flex flexDirection="column" alignItems="center" padding={32}>
      {result.data && <DataTableV2 data={result.data?.records} columns={convertToColumnsV2(result.data?.types)} />}
    </Flex>
  );
};

Tip

This operation requires the scope storage:files:read. Read more about scopes in this guide.

Handling user permission

Most users may not have permission to write lookup data or may only have access to specific folders. Applications must gracefully handle such restrictions, allowing users to choose where to store data within the customer-managed /lookups/ directory. To respect user control and permissions, avoid creating predefined structures like /lookups/dt/.

In the future, Dynatrace may introduce a /dt/ prefix for structured data management and enhanced permission handling. This feature is under development and will align with evolving use cases.