Paging

nestjs-query supports multiple paging strategies that each have their own pros and cons. This documentation will cover the different paging strategies and their applicable use cases. The following examples are based on the following TodoItemDTO

todo-item.dto.ts

import { FilterableField, IDField } from '@ptc-org/nestjs-query-graphql'
import { ObjectType, ID, GraphQLISODateTime } from '@nestjs/graphql'

@ObjectType('TodoItem')
export class TodoItemDTO {

  @IDField(() => ID)
  id!: string

  @FilterableField()
  title!: string

  @FilterableField()
  completed!: boolean

  @FilterableField(() => GraphQLISODateTime)
  created!: Date

  @FilterableField(() => GraphQLISODateTime)
  updated!: Date
}

Cursor Based Paging

By default nestjs-query will expose all query many endpoints as cursor based connections that you can use to page through results.

When using cursor based connections you are not tied to any particular implementation described below, because of the opaque nature of cursors you can start out with the default and switch to key set based cursors later on without changing your clients.

All cursor connections, regardless of paging strategy, expose the following schema

scalar ConnectionCursor

type PageInfo {
  hasNextPage: Boolean
  hasPreviousPage: Boolean
  startCursor: ConnectionCursor
  endCursor: ConnectionCursor
}

type TodoItemConnection {
  pageInfo: PageInfo!
  edges: [TodoItemEdge!]!
}

type TodoItemEdge {
  node: TodoItem!
  cursor: ConnectionCursor!
}

type TodoItem {
  id: ID!
  title: String!
  description: String
  completed: Boolean!
  created: DateTime!
  updated: DateTime!
}

Offset Based Cursor

By default all cursors will use a form of offset based paging to back cursors.

Pros	Cons
Allow for recursive relation paging	Pages and nodes may change between queries
Straight forward to implement	Inconsistent Sorting - natural sorting is used by default
	Queries grow more inefficient as you go through more pages

Key Set Based Cursor

You have the option to specify a key set on your DTO which will replace the offset with a where clause. A keyset is a set of fields that uniquely identify a record (e.g. id).

Pros	Cons
Consistent Results - All cursors uniquely identify a record allowing for deterministic before/after pages	Unable to do recursive paging for relations
Stable Sorting - All results will be deterministically sorted	A unique key set must be specified
Consistent performance with correct indexes in place	Unable to jump to arbitrary page
Suitable for datasets of any size

To enable key set based paging all you need to do is decorate your DTO with the @KeySet decorator.

import { FilterableField, IDField, KeySet } from '@ptc-org/nestjs-query-graphql'
import { ObjectType, ID, GraphQLISODateTime } from '@nestjs/graphql'

@ObjectType('TodoItem')
@KeySet(['id'])
export class TodoItemDTO {

  @IDField(() => ID)
  id!: number

  @FilterableField()
  title!: string

  @FilterableField({ nullable: true })
  description?: string

  @FilterableField()
  completed!: boolean

  @FilterableField(() => GraphQLISODateTime)
  created!: Date

  @FilterableField(() => GraphQLISODateTime)
  updated!: Date
}

Sorting and Key set cursors

When using key set based cursors we must take into account any client provided sorting in order to uniquely identify a record within the sorted set of nodes For example assume we’re using the same DTO as above. If we added a sort by completed a comparision on id would no longer be sufficient

{
  todoItems(sorting: [{ field: completed, direction: ASC }]) {
    edges {
      cursor
      node {
        id
        title
        completed
      }
    }
  }
}

If we only compared on the keyset [id] our pages would no longer be sorted properly, if we only compared on completed you would not be able to page. We solve this problem by encoding information about the each field in the sort as well as the key set fields into the cursor so we can page properly. In the above example the filter from the cursor (when paging forward) would be something like

WHERE (completed > ? OR (completed = ? AND id > ?))

Relation Connections

Key set paging will not apply to relations because they are recursive by nature. For example if you query for multiple TodoItems and their subTasks if key set paging was used for the subTasks connection the cursor from one todoItems subTasks may not be applicable to all todoItems For example, assume you have the following todo items and subTasks

[
  {
    "id": 1,
    "title": "Todo 1"
    "subTasks": [
      {"id": 1, "title": "Todo 1 - Sub Tasks 1"}, // cursor: "abc"
      {"id": 2, "title": "Todo 1 - Sub Tasks 2"}  // cursor: "def"
    ]
  },
  {
    "id": 2,
    "title": "Todo 2"
    "subTasks": [
      {"id": 3, "title": "Todo 2 - Sub Tasks 1"},  // cursor: "ghi"
      {"id": 4, "title": "Todo 2 - Sub Tasks 2"}  // cursor: "jkl"
    ]
  }
]

If you ran the following graphql query

{
  todoItems {
    edges {
      node {
        title
        subTasks(paging: { first: 2, after: "ghi" }) {
          title
        }
      }
    }
  }
}

The resulting query would look for all subTasks with an id > 3 breaking paging for Todo 1, for this reason the @KeySet decorator is used for all relations.

To page with cursors it works the same way for all strategies. In this example we’ll fetch the first 2 records.

GraphQL
Response

{
  todoItems(paging: { first: 2 }) {
    pageInfo {
      hasNextPage
      hasPreviousPage
      startCursor
      endCursor
    }
    edges {
      node {
        id
        title
        completed
        created
        updated
      }
      cursor
    }
  }
}

{
  "data": {
    "todoItems": {
      "pageInfo": {
        "hasNextPage": true,
        "hasPreviousPage": false,
        "startCursor": "YXJyYXljb25uZWN0aW9uOjA=",
        "endCursor": "YXJyYXljb25uZWN0aW9uOjE="
      },
      "edges": [
        {
          "node": {
            "id": "1",
            "title": "Create One Todo Item",
            "completed": false,
            "created": "2020-01-14T07:00:31.763Z",
            "updated": "2020-01-14T07:00:31.763Z"
          },
          "cursor": "YXJyYXljb25uZWN0aW9uOjA="
        },
        {
          "node": {
            "id": "2",
            "title": "Create Many Todo Items - 1",
            "completed": false,
            "created": "2020-01-14T07:00:34.111Z",
            "updated": "2020-01-14T07:00:34.111Z"
          },
          "cursor": "YXJyYXljb25uZWN0aW9uOjE="
        }
      ]
    }
  }
}

Lets take a look the pageInfo from the previous example

{
  "pageInfo": {
    "hasNextPage": true,
    "hasPreviousPage": false,
    "startCursor": "YXJyYXljb25uZWN0aW9uOjA=",
    "endCursor": "YXJyYXljb25uZWN0aW9uOjE="
  }
}

Notice how hasNextPage is true and there is an endCursor that can be used to fetch the next page.

GraphQL
Response

{
  todoItems(paging: { after: "YXJyYXljb25uZWN0aW9uOjE=", first: 2 }) {
    pageInfo {
      hasNextPage
      hasPreviousPage
      startCursor
      endCursor
    }
    edges {
      node {
        id
        title
        completed
        created
        updated
      }
      cursor
    }
  }
}

{
  "data": {
    "todoItems": {
      "pageInfo": {
        "hasNextPage": true,
        "hasPreviousPage": false,
        "startCursor": "YXJyYXljb25uZWN0aW9uOjI=",
        "endCursor": "YXJyYXljb25uZWN0aW9uOjM="
      },
      "edges": [
        {
          "node": {
            "id": "3",
            "title": "Create Many Todo Items - 2",
            "completed": true,
            "created": "2020-01-14T07:00:34.111Z",
            "updated": "2020-01-14T07:00:34.111Z"
          },
          "cursor": "YXJyYXljb25uZWN0aW9uOjI="
        },
        {
          "node": {
            "id": "4",
            "title": "Create Many Todo Items - 3",
            "completed": false,
            "created": "2020-01-14T07:01:27.805Z",
            "updated": "2020-01-14T07:01:27.805Z"
          },
          "cursor": "YXJyYXljb25uZWN0aW9uOjM="
        }
      ]
    }
  }
}

We can also page backward by using before and last. In the following example we’ll use the startCursor from the previous example and set last to 2.

GraphQL
Response

{
  todoItems(paging: { before: "YXJyYXljb25uZWN0aW9uOjI=", last: 2 }) {
    pageInfo {
      hasNextPage
      hasPreviousPage
      startCursor
      endCursor
    }
    edges {
      node {
        id
        title
        completed
        created
        updated
      }
      cursor
    }
  }
}

{
  "data": {
    "todoItems": {
      "pageInfo": {
        "hasNextPage": false,
        "hasPreviousPage": false,
        "startCursor": "YXJyYXljb25uZWN0aW9uOjA=",
        "endCursor": "YXJyYXljb25uZWN0aW9uOjE="
      },
      "edges": [
        {
          "node": {
            "id": "1",
            "title": "Create One Todo Item",
            "completed": false,
            "created": "2020-01-14T07:00:31.763Z",
            "updated": "2020-01-14T07:00:31.763Z"
          },
          "cursor": "YXJyYXljb25uZWN0aW9uOjA="
        },
        {
          "node": {
            "id": "2",
            "title": "Create Many Todo Items - 1",
            "completed": false,
            "created": "2020-01-14T07:00:34.111Z",
            "updated": "2020-01-14T07:00:34.111Z"
          },
          "cursor": "YXJyYXljb25uZWN0aW9uOjE="
        }
      ]
    }
  }
}

Offset Based Paging

An alternative to cursor based querying is to use the OFFSET pagingStrategy. When using the OFFSET strategy queries that return multiple records will return an OffsetConnection that looks like the following.

type OffsetPageInfo {
  hasNextPage: Boolean
  hasPreviousPage: Boolean
}

type TodoItemConnection {
  pageInfo: OffsetPageInfo!
  nodes: [TodoItem!]!
}

type TodoItem {
  id: ID!
  title: String!
  description: String
  completed: Boolean!
  created: DateTime!
  updated: DateTime!
}

Pros	Cons
Easy to understand	unstable - pages and nodes may change between queries
Able to jump to arbitrary pages	Inconsistent Sorting - natural sorting is used in the strategy
	Queries grow more inefficient as you go through more pages
	State must be maintained in the client to know last limit and offset in order to page

To enable OFFSET based paging all you need to do is set the pagingStrategy option using the @QueryOptions decorator.

todo-item.dto.ts

import { FilterableField, IDField, QueryOptions } from '@ptc-org/nestjs-query-graphql'
import { ObjectType, ID, GraphQLISODateTime } from '@nestjs/graphql'

@ObjectType('TodoItem')
@QueryOptions({ pagingStrategy: PagingStrategies.OFFSET })
export class TodoItemDTO {

  @IDField(() => ID)
  id!: number

  @FilterableField()
  title!: string

  @FilterableField({ nullable: true })
  description?: string

  @FilterableField()
  completed!: boolean

  @FilterableField(() => GraphQLISODateTime)
  created!: Date

  @FilterableField(() => GraphQLISODateTime)
  updated!: Date
}

In this example we’ll fetch the first 2 records.

GraphQL
Response

{
  todoItems(paging: { limit: 2 }) {
    pageInfo {
      hasNextPage
      hasPreviousPage
    }
    nodes {
      id
      title
      completed
      created
      updated
    }
  }
}

{
  "data": {
    "todoItems": {
      "pageInfo": {
        "hasNextPage": true,
        "hasPreviousPage": false
      },
      "nodes": [
        {
          "id": "1",
          "title": "Create Nest App",
          "completed": true,
          "created": "2021-02-14T12:56:01.106Z",
          "updated": "2021-02-14T12:56:01.106Z"
        },
        {
          "id": "2",
          "title": "Create Entity",
          "completed": false,
          "created": "2021-02-14T12:56:01.106Z",
          "updated": "2021-02-14T12:56:01.106Z"
        }
      ]
    }
  }
}

In this example we’ll also pass in an offset to fetch the next 2 records.

GraphQL
Response

{
  todoItems(paging: { limit: 2, offset: 2 }) {
    pageInfo {
      hasNextPage
      hasPreviousPage
    }
    nodes {
      id
      title
      completed
      created
      updated
    }
  }
}

{
  "data": {
    "todoItems": {
      "pageInfo": {
        "hasNextPage": true,
        "hasPreviousPage": true
      },
      "nodes": [
        {
          "id": "3",
          "title": "Create Entity Service",
          "completed": false,
          "created": "2021-02-14T12:56:01.106Z",
          "updated": "2021-02-14T12:56:01.106Z"
        },
        {
          "id": "4",
          "title": "Add Todo Item Resolver",
          "completed": false,
          "created": "2021-02-14T12:56:01.106Z",
          "updated": "2021-02-14T12:56:01.106Z"
        }
      ]
    }
  }
}

Disabling Paging

This strategy is only recommended if you are sure your dataset is small.

When using the NONE paging strategy the paging argument is removed and all methods will return an ArrayConnection.

Pros	Cons
All data is returned at once	Not suitable for large datasets

To disable paging all you can set the pagingStrategy option using the @QueryOptions decorator to PagingStrategies.NONE.

todo-item.dto.ts

import { FilterableField, IDField, QueryOptions, PagingStrategies } from '@ptc-org/nestjs-query-graphql'
import { ObjectType, ID, GraphQLISODateTime } from '@nestjs/graphql'

@ObjectType('TodoItem')
@QueryOptions({ pagingStrategy: PagingStrategies.NONE })
export class TodoItemDTO {

  @IDField(() => ID)
  id!: number

  @FilterableField()
  title!: string

  @FilterableField({ nullable: true })
  description?: string

  @FilterableField()
  completed!: boolean

  @FilterableField(() => GraphQLISODateTime)
  created!: Date

  @FilterableField(() => GraphQLISODateTime)
  updated!: Date
}

When using the strategy queries that return multiple records will return an array instead of a connection.

GraphQL
Response

{
  todoItems {
    id
    title
    completed
    created
    updated
  }
}

{
  "data": {
    "todoItems": [
      {
        "id": "1",
        "title": "Create Nest App",
        "completed": true,
        "created": "2020-06-12T08:15:18.876Z",
        "updated": "2020-06-12T08:15:18.876Z"
      },
      {
        "id": "2",
        "title": "Create Entity",
        "completed": false,
        "created": "2020-06-12T08:15:18.876Z",
        "updated": "2020-06-12T08:15:18.876Z"
      },
      {
        "id": "3",
        "title": "Create Entity Service",
        "completed": false,
        "created": "2020-06-12T08:15:18.876Z",
        "updated": "2020-06-12T08:15:18.876Z"
      },
      {
        "id": "4",
        "title": "Add Todo Item Resolver",
        "completed": false,
        "created": "2020-06-12T08:15:18.876Z",
        "updated": "2020-06-12T08:15:18.876Z"
      },
      {
        "id": "5",
        "title": "How to create item With Sub Tasks",
        "completed": false,
        "created": "2020-06-12T08:15:18.876Z",
        "updated": "2020-06-12T08:15:18.876Z"
      }
    ]
  }
}

​Cursor Based Paging

​Offset Based Cursor

​Key Set Based Cursor

​Sorting and Key set cursors

​Relation Connections

​Paging

​Offset Based Paging

​Disabling Paging

Cursor Based Paging

Offset Based Cursor

Key Set Based Cursor

Sorting and Key set cursors

Relation Connections

Paging

Offset Based Paging

Disabling Paging