URI-style Cursor Pagination

Some APIs return a complete URL path for the next page instead of just a cursor value.

When to Use

• API returns a full path like /2010-04-01/Messages.json?Page=1
• API returns an absolute URL for the next page
• Cursor value includes query parameters

Configuration

{
  enabled: true,
  type: "cursor",
  cursorIsFullPath: true,  // KEY SETTING
  params: {
    limitParam: "PageSize",
    defaultLimit: 100
  },
  response: {
    nextCursorPath: "next_page_uri",
    dataArrayPath: "messages"
  },
  stopCondition: {
    type: "cursor_empty"
  }
}

Example: Twilio API

API Behavior

First Request:

GET /2010-04-01/Accounts/AC123/Messages.json?PageSize=100

Response:

{
  "messages": [
    { "sid": "SM123", "body": "Hello" },
    { "sid": "SM456", "body": "World" }
  ],
  "next_page_uri": "/2010-04-01/Accounts/AC123/Messages.json?Page=1&PageToken=abc",
  "page": 0,
  "page_size": 100
}

Second Request (uses full path):

GET /2010-04-01/Accounts/AC123/Messages.json?Page=1&PageToken=abc

Configuration

{
  type: "cursor",
  cursorIsFullPath: true,
  params: {
    limitParam: "PageSize",
    defaultLimit: 100
  },
  response: {
    nextCursorPath: "next_page_uri",
    dataArrayPath: "messages"
  },
  stopCondition: {
    type: "cursor_empty"
  }
}

How It Works

1. First request: Normal request to endpoint path
2. Response: Contains next_page_uri with complete path
3. Second request: Uses entire next_page_uri as the request path
4. Continues: Until next_page_uri is null or missing

Request Flow

Request 1: GET /api/messages?limit=100
           ↓
Response:  { "next_page_uri": "/api/messages?page=2&token=abc" }
           ↓
Request 2: GET /api/messages?page=2&token=abc
           ↓
Response:  { "next_page_uri": "/api/messages?page=3&token=def" }
           ↓
Request 3: GET /api/messages?page=3&token=def
           ↓
Response:  { "next_page_uri": null }  // STOP

Example: Salesforce API

API Behavior

First Request:

GET /services/data/v55.0/query?q=SELECT+Id+FROM+Account

Response:

{
  "totalSize": 5000,
  "done": false,
  "nextRecordsUrl": "/services/data/v55.0/query/01gD0000002HU6KIAW-2000",
  "records": [...]
}

Configuration

{
  type: "cursor",
  cursorIsFullPath: true,
  params: {
    defaultLimit: 2000
  },
  response: {
    nextCursorPath: "nextRecordsUrl",
    dataArrayPath: "records"
  },
  stopCondition: {
    type: "cursor_empty"
  }
}

Key Differences from Standard Cursor

Aspect	Standard Cursor	URI-style Cursor
Cursor value	Token like abc123	Full path like /api?page=2
Request building	Add cursor as parameter	Use cursor as entire path
cursorIsFullPath	false (default)	true

Handling Absolute URLs

The cursor value may be either an absolute URL or a relative path:

{
  "next": "https://api.example.com/v1/users?page=2"
}

• If the cursor starts with http, it is used verbatim as the next request URL (the connector base URL is ignored for that request).
• Otherwise it is treated as a path and appended to the connector base URL (a leading / is added if missing).

So an absolute https://api.example.com/v1/users?page=2 is requested exactly as returned — it is not stripped down to /v1/users?page=2.

Troubleshooting

Path Not Being Used Correctly

Check:

• cursorIsFullPath is set to true
• Cursor path returns the full URI
• No extra manipulation of cursor value

Base URL Conflicts

Check:

• For relative cursors, the connector base URL is prepended — make sure that base URL is correct
• For absolute (http...) cursors, the URL is used as-is, so the host must be reachable and correct in the returned value
• Verify request logs show correct URLs

Query Parameters Duplicated

Symptoms: Parameters appear twice in URL

Check:

• Don't add limit param if cursor includes it
• System uses cursor path as-is when cursorIsFullPath is true