Filtering and sorting

Introduction

One common feature most services provide is the ability to return a list of resources through an API. These resources can be filtered and sorted based on various criteria.

Let's consider the examples of the App Engine Registry and Documents APIs. The /apps endpoint enables you to retrieve a list of all installed applications, while the /documents endpoint returns a list of all available documents, as shown below:

By utilizing the filtering and sorting options provided by these APIs, you can easily narrow your search and retrieve only the relevant information to your needs.

Sorting

You can use the query parameter sort to sort the list. The sort parameter requires a single field name or a comma-separated list of field names defining the sorting order. The order of the field names in the list is left to right, meaning that the first field name is the primary sorting criteria, followed by the second, and then next fields.

By default, the sorting order is ascending. However, you can change this by prefixing the field name with a minus sign, -. It will sort the list in descending order based on that particular field. It's important to note that string comparison for sorting is case-insensitive.

Look at the following API call:

GET …/problems?sort=status,-startTime,relevance

The API will sort the result items by ascending status first, then by descending startTime, and finally ascending relevance.

Filtering

You can use the query parameter filter if the API supports filtering. By default, the filter expression can reference any field in the listed resource. However, some services may restrict the fields that you can filter.

The filtered result is always a list regardless of the number of entries. If the result is empty, it's still considered a successful execution of the API with an HTTP 200 - Ok response code.

Construct filter expression

To construct a filter expression, use a set of field-level expressions combined with boolean operators. The boolean operators supported are or, and, and not.

You can also use parentheses to group expressions. If you don't use parentheses, the standard boolean operator precedence is applied, with and taking precedence over or.

The general form of the field-level expression is <Fieldname> <Operator> <Value>. Let's understand them:

• <Fieldname> is the field name you want to filter.
• <Operator> is an operator that specifies how the field should be in comparison to the specified value.
• <Value> is the value against which the field is compared. Depending on the filtered field type, the value can be a string, date/time, number, or boolean.

Datatypes and operators

Following is the list of data types supported in the filter expression along with the operators:

Datatype	Supported operators	Representation
number (short, int, long, float, double)	=, !=, < , <=, >, >=, in	Integers: decimal and hexadecimal (with leading 0x) Floating Point: scientific notation with optional exponent e or E
String	=, !=, contains, starts-with, ends-with, in	Single quotes only: 'Hello World!' Special characters (for example, the quotes) are preceded with \ exact operators = and != are case sensitive inexact operators contains, starts-with, and ends-with are case insensitive
Boolean	=, !=	Comparison with constants true or false only
Date/Time	=, !=, < , <=, >, >=, in	As ISO 8601 compliant string in one of the following formats: yyyy-MM-ddThh:mm:ss[+/-]hh:mm (e.g. 2007-12-03T10:15:30+01:00) yyyy-MM-ddThh:mm:ss (e.g. 2007-12-03T10:15:30) yyyy-MM-dd (e.g. 2007-12-03) hh:mm:ss (e.g. 10:15:30)

The 'in' operator

It can compare for equality within a list of possible values provided by the 'in' operator.

So instead of writing

s = '123' or s = '456' or s = '789'

you can write

s in('123','456','789')

The 'in' operator supports all data types except boolean.

Examples

GET /clients?filter=age=30
GET /clients?filter=firstName='Konrad' and lastName='Zuse'
GET /documents?filter=owner='user1' and lastModified>='2022-02-06T11:00:00Z'
GET /tenants?filter=tenantUuid starts-with 'abc' and not(deleted=false or active=true)
GET /caches?filter=cacheHitRate<90.5
GET /locations?filter=distance>=1.0E4

Field filtering and partial results

Some services may return only a subset of all available fields of a resource by default, also known as a partial result. This can be beneficial in certain scenarios to avoid expensive background operations and unnecessary network bandwidth usage.

If an API supports partial results, it accepts a query parameter add-fields to include missing fields in the default response.

Field filtering is only supported in the APIs that list or get a resource.

Things to remember

• add-fields accept a comma-separated list of field names that are added to the default set of fields.
• Duplicate fields in the list result in an error.
• Adding fields already in the default response is considered redundant and ignored.
• Referencing unknown fields results in an error.
• A dot separates nested field names.

Example

In the following example, you're fetching a list of entities. The result includes default fields.

GET /entities

{
  "totalCount": 72,
  "nextPageKey": "…",
  "entities": [
    {
      "entityId": "HOST-0004DD30F142D18C"
    }
  ]
}

In the following example, you're using add-fields to add lastSeenTms and properties.bitness in addition to the default result, which you can see in the result below:

GET /entities?add-fields=lastSeenTms,properties.bitness

{
  "totalCount": 72,
  "nextPageKey": "…",
  "entities": [
    {
      "entityId": "HOST-0004DD30F142D18C",
      "lastSeenTms": 1615991063257,
      "properties": {
        "bitness": "64"
      }
    }
  ]
}