Skip to main content

Storage service

The storage services provide APIs to store and query data, such as: events, logs, and metrics.


You need to know the following concepts to work with the storage service.

Grail query API

The Grail Query API is the entry point for querying data stored in Grail.

You can query data in Grail using the . To write queries, you use Dynatrace Query Language (DQL).

There are two types of operations that the Grail Query API do:

  • Query execution - Lets the client start queries and fetch the results or cancel a running query.
  • Language services - Support writing DQL queries and providing more syntactic information.

Query execution

Queries are executed asynchronously via REST.

The final result is guaranteed to be available for one minute after the query has finished, so a polling interval of less than one minute is strongly recommended.

Query start

By sending a HTTP POST request, the query is started. The query and all corresponding parameters are part of the request body in JSON. The response will contain a request-token as a query identifier for the following requests.

Polling and long polling

You can use request-token to poll queries. The response for a polling request contains the status of the query.

Long polling

Starting a query and polling offer the possibility to specify a request-timeout. If the final result of a preview is available before the specified timeout, the API returns the response; otherwise, it returns the state of the query.


It's possible to enable previews with the initial request. Following polling responses will contain new previews, if available. A new preview will always be whole, not as a delta to any previous preview.


You can cancel a query using the request-token from the response to the initial request. If the query has finished, you'll get the final result as a response to the cancel request. Otherwise, Grail will stop the execution of the query and discard the result.


Metadata is part of the query response next to the query result. Some metadata types are always added, whereas others must be requested explicitly by setting the enrich parameter to specify the list of the requested metadata types.

The following metadata types are provided:

grailThe Grail metadata is always added and gives meta-information about the query execution.
metricsMetric metadata can be requested by adding the parameter ?enrich=metric-metadata to /query:execute and /query:poll requests. Suppose the query result contains metric keys (for example, ). In that case, the response will be enriched with an additional metadata section, metrics, which contains information about those metrics, such as their units and display names.

These are the endpoints for starting queries, fetching the current status, including the final result and query cancellation.

/query:executeStarts a Grail query.
/query:pollRetrieves the query status and final result from Grail.
/query:cancelCancels the query and returns the result if the query has finished. Otherwise, the query and its result are discarded.

Grail language services

The language services provide support when writing DQL queries.

/query:verifyVerifies a DQL query without executing it. If the query contains errors, the response will have corresponding notifications.
/query:parseCreates a tree of the canonical form of the query. Nodes contain references to the related token positions in the DQL query string. It can be used for hover effects, marking optional items, displaying canonical forms, and more.
/query:autocompleteCreates a list of suggestions for the query at the given cursor position.


You need to have the correct permissions to query data stored in Grail. All records are stored in Grail buckets and each bucket is assigned to a table.

Bucket Permissions

A typical bucket read permission looks like ALLOW storage:buckets:read;. Adding this permission grants access to all the buckets. Note that you need table permissions in addition to this to access Grail data. Without proper table permissions, it will still not be accessible.

Bucket permissions can also be conditional depending on what buckets are needed to be made accessible. The following conditional permission provides access to buckets with the name default_logs and default_metrics.

ALLOW storage:buckets:read WHERE storage:bucket-name IN ("default_logs", "default_metrics");

Currently, =, IN, STARTSWITH operators are supported in conditions.

Table Permissions

There are predefined tables in Grail, and permissions on those tables you can define as:

ALLOW storage:metrics:read;, ALLOW storage:logs:read;, ALLOW storage:events:read;, ALLOW storage:system:read;, etc.

You can add table permissions to grant access to the table, but you need to combine them with bucket permissions to access data stored in the buckets. A few tables have unrestricted access and don't need bucket or table permissions, such as dt.system.data_objects.

Record Level Permissions

You can extend the table permissions by adding conditions on some predefined fields. Adding those conditions grants access to only those records in the table that satisfy the condition.

The following conditional table permission, a.k.a. record level permission, grants access to only those records in the metrics table that have metric key

ALLOW storage:metrics:read WHERE storage:metric.key = "";

Currently, =, IN, and STARTSWITH operators are supported in conditions.

You can join the conditions by the AND operator.

ALLOW storage:logs:read WHERE storage:log.source STARTSWITH "/var/log" AND IN ("host1", "host2");

If there are multiple statements for the same table, they're implicitly joined by OR. The following set of conditional table permissions grant access to those records in the events table that have an event type of PROCESS_RESTART OR an event kind of DAVIS_EVENT.

ALLOW storage:events:read WHERE storage:event.type = "PROCESS_RESTART";
ALLOW storage:events:read WHERE storage:event.kind = "DAVIS_EVENT";

Unconditional table permission overrides conditional table permissions on the same table. The following set of permissions grants access to all records in the events table:

ALLOW storage:events:read WHERE storage:event.type = "PROCESS_RESTART";
ALLOW storage:events:read;

As with table permissions, you need to combine them with bucket permissions to access data stored in the buckets.

EndpointPermission Handling
/query:executeRequires bucket and table permissions.
/query:pollPermissions of execute request apply. Only the user that started the query can poll.
/query:cancelPermissions of execute request apply. Only the user that started the query can cancel.
/query:verifyRequires table permissions.
/query:parseRequires table permissions.
/query:autocompleteRequires bucket and table permissions.
Still have questions?
Find answers in the Dynatrace Community