Pagination

Cursor pagination

Collection endpoints return one page plus an opaque cursor pointing at the next.

GET /models?limit=2

{
  "items": [
    { "id": "ast_9Qd...", "slug": "bin-picking-policy-v2" },
    { "id": "ast_7Kf...", "slug": "warehouse-depth-set" }
  ],
  "next_cursor": "ast_7Kf...",
  "has_more": true
}

Why not ?page=2

Offset pagination is anchored to a position in a result set that keeps changing. Publish one listing between requests and everything shifts down by one — so the item that was last on page 1 becomes first on page 2, and you either process it twice or skip it entirely. A cursor is anchored to a record, so insertions elsewhere don't move it.

Walking every page by hand

walk.ts

1let cursor: string | undefined;
2 
3do {
4  const url = new URL("https://www.nodedata.ai/api/v1/models");
5  url.searchParams.set("limit", "100");
6  if (cursor) url.searchParams.set("cursor", cursor);
7 
8  const res = await fetch(url, {
9    headers: { authorization: `Bearer ${key}` },
10  });
11  if (!res.ok) throw new Error(`${res.status}`);
12 
13  const page = await res.json();
14  for (const asset of page.items) await process(asset);
15 
16  cursor = page.next_cursor ?? undefined;
17} while (cursor);

Or let the SDK do it

A Page is both the first page and an async iterable over all of them. Iterating fetches lazily, one page at a time, so memory stays flat regardless of collection size.

sdk.ts

const page = await nd.models.list({ type: "dataset", limit: 100 });
 
page.items;      // Asset[] — just this page
page.hasMore;    // boolean
page.nextCursor; // string | null
 
// Walk everything, lazily
for await (const asset of page) {
  await process(asset);
}
 
// Or collect with a ceiling — always set one on an unbounded collection
const first500 = await page.all(500);
 
// One page at a time, explicitly
const second = await page.next(); // Page<Asset> | null

Resuming after a crash

Cursors are opaque strings, so you can persist one and pick up where you left off in a later process. This is the pattern for any long sync.

resumable.ts

let cursor = await store.get("sync:models:cursor");
 
while (true) {
  const page = await nd.models.list({ limit: 100, cursor });
  for (const asset of page.items) await upsert(asset);
 
  // Checkpoint after the page is fully processed, not before.
  if (!page.nextCursor) break;
  cursor = page.nextCursor;
  await store.set("sync:models:cursor", cursor);
}
 
await store.del("sync:models:cursor");

Checkpoint after processing, not after fetching

Saving the cursor before the page is handled means a crash silently skips a page. Saving it after means a crash reprocesses one page — which is fine if your writes are upserts, and it's why they should be.

Limits

Endpoint	Default	Max
`GET /models`	20	100
`GET /payouts`	50	200

Over-large values are clamped rather than rejected, so a limit=5000 quietly becomes the maximum. Don't rely on getting the number you asked for — always follow the cursor.