The Noded Provider SDK

How it works

A provider is an Express server that you host. The Noded platform calls it over HTTPS. You never call the platform yourself, with one small exception during sign-in. Five facts explain almost everything on this page:

• The platform calls you. Noded schedules syncs, retries failures, and pages through results. Your job is only to answer each request.
• You store nothing. Your provider keeps no database and no tokens. Every request body carries the user's credential. When a user first connects, you hand their credential to the platform once, and the platform sends it back to you on every later call.
• Data is pulled on a schedule. Most sync types run every few minutes. There is no webhook path in the SDK - polling plus fast cursor-driven pagination covers it.
• You translate, Noded reconciles. Your handlers turn vendor JSON into Noded's primitive types (transcriptions, emails, tables, tags). The platform matches each item by its externalId and decides what to create, update, or archive.
• One shared secret secures the channel. The platform authenticates to you with Authorization: Bearer using your BIGFOOT_API_KEY, and you authenticate to the platform with the same key.

You build all of this by passing plain functions to startServer(). Every group is optional - implement only what your tool supports. A meeting recorder might implement auth and sync.transcriptions and nothing else. That is a complete, useful provider.

import { startServer } from '@bigfootai/sdk';

await startServer({
  auth: {
    // how a user connects their account
    connect, callback, check, refresh,
  },
  sync: {
    // how data gets pulled in
    transcriptions,
  },
});

interface StartServerOptions {
  auth?:     Partial<AuthMethods>;
  sync?:     Partial<SyncMethods>;
  link?:     Partial<LinkMethods>;
  tag?:      Partial<TagMethods>;
  upsert?:   Partial<UpsertMethods>;
  function?: Partial<FunctionMethods>;
  query?:    Partial<QueryMethods>;
}

Your first provider

Start from the template. It is a Copier template that asks for your provider's name and generates a runnable project: a stub src/index.ts, TypeScript and Biome config, a Dockerfile, and a CI workflow.

Three environment variables wire your provider to the platform. Put them in .env for local work:

The SDK gives you a lot for free, so your code stays focused on the vendor API:

• Route registration and request validation for every handler you provide.
• Platform authentication middleware on every route except /auth/connect, /auth/callback, and the health check.
• Structured logging (pino) with credential redaction built in. Tokens, keys, and secrets never reach your logs, even if you log a whole request body.
• JSON body parsing (up to 50 MB), a health check at GET /api/v1/hello, static file serving from ./public (put your logo there), and a central error handler.

startServer() returns the Express app, so you can attach extra routes after it - a custom form endpoint, for example. The complete example below shows a full working provider.

copier copy gh:Notify-AI/bigfoot-sdk-template bigfoot-skylark
cd bigfoot-skylark
pnpm install
cp .env.example .env   # fill in the three variables
pnpm dev

curl http://localhost:5000/api/v1/hello
# -> You look nice today :)

BASE_URL=https://connect-skylark.example.com
BIGFOOT_API_KEY=nk_...        # from Noded
BIGFOOT_ENVIRONMENT_URL=https://api.getnoded.ai

Errors

Every error your provider returns is an application/problem+json response (RFC 7807). You do not need catch-all try/catch blocks: anything you throw is caught by the SDK's error handler, logged, and returned as a structured 500. But two errors deserve to be thrown on purpose, because the platform reacts to them:

For anything more specific, build a problem yourself with ProblemDetailsBuilder from problem-details-http - for example, mapping a vendor 404 or 429 to the matching status. And for problems that should not fail the whole sync, do not throw at all: return your items and put a note in metadata.warnings instead.

import {
  CredentialsExpiredError,
  ValidationError,
} from '@bigfootai/sdk';

if (response.status === 401) {
  throw new CredentialsExpiredError(
    'Skylark rejected the access token',
  );
}

if (!filter.dateStart) {
  throw new ValidationError(
    'filter.dateStart must not be null',
  );
}

{
  "type": "https://getnoded.ai/credentials-expired",
  "title": "Credentials expired",
  "status": 401,
  "detail": "Skylark rejected the access token"
}

return {
  items: transcriptions,
  metadata: {
    warnings: [
      'Dropped 2 of 40 meetings: no transcript yet.',
    ],
  },
  pagination: { hasMore: false, pageSize: 38 },
};

Authentication: the connect flow

When a user clicks your integration in Noded, a small dance runs between the platform, your provider, and the vendor. It has four steps, and each step is one handler:

1. The platform opens GET /auth/connect in a popup. Your auth.connect answers with one of two things: a redirect to the vendor's OAuth page, or an HTML form where the user pastes an API key. Both shapes are the union type AuthConnectResult.
2. The user signs in with the vendor, which sends them back to GET /auth/callback. Your auth.callback exchanges the code (or validates the key) and returns a ConnectedProviderRegistration - the connection ID plus a Credential.
3. If you implement auth.check, the SDK runs it right away. If the user granted too few scopes, the popup shows a friendly error page instead of completing.
4. The SDK posts the registration to the platform (this is the one time your provider calls Noded), and the popup closes. From now on, every platform request carries that credential back to you.

One convention carries the context through the round trip: pack the connection ID and application into the OAuth state parameter as `${connectionId}.${application}`. Your callback splits it back apart, and the SDK uses it to run auth.check with the right application.

Later, on a schedule, the platform calls auth.refresh to keep tokens fresh, and may call auth.check again to spot revoked scopes early.

type AuthConnectResult =
  | { type: 'redirect'; url: string }   // OAuth
  | { type: 'html'; html: string };     // key form

interface Credential {
  accessToken: string;
  refreshToken?: string;
  dateExpiry?: number;    // epoch ms
  email: string;          // the user at the vendor
  instanceId?: string;
  instanceUrl?: string;   // per-tenant vendor host
  profile: unknown;       // anything useful to keep
}

GET  /auth/connect    # public
GET  /auth/callback   # public
POST /auth/check      # platform-authenticated
POST /auth/refresh    # platform-authenticated

Concept: providers and applications

Your service is one provider - skylark, say. A provider offers one or more applications: the tiles a user actually connects in Noded. Google is one provider with several applications (calendar, email, drive); a meeting tool is usually one provider with one application.

The provider definition is what registers your service with Noded. It says where your provider lives (instanceUrl - the base URL the platform calls) and, for each application, which capabilities it supports. The sobjects flags are the important part: each flag you turn on is a promise that the matching handler exists. Turn on transcription: true and the platform will start posting to your /sync/transcription.

The application name is also the thread that runs through everything: it arrives in auth.connect, travels inside the OAuth state, names the /:application/link route, and comes back in every credential's connectedProvider.application. Guard it in your handlers - reject names you do not serve.

{
  "name": "skylark",
  "label": "Skylark",
  "description": "Skylark meetings for Noded",
  "instanceUrl": "https://connect-skylark.example.com",
  "iconUrl": "https://connect-skylark.example.com/images/logo.svg",
  "applications": [
    {
      "name": "calls",
      "label": "Skylark Calls",
      "description": "Ingest Skylark meeting recordings as transcripts in Noded",
      "authentication": true,
      "sobjects": {
        "calendar": false,
        "transcription": true,
        "message": false,
        "email": false,
        "table": false,
        "function": false
      },
      "linkSubscriptions": [],
      "externalIds": []
    }
  ]
}

Concept: primitives

Everything Noded stores is a typed object from @bigfootai/bigfoot-types. Your handlers produce these primitives, and each sync handler produces exactly one kind:

Most of these extend BusinessObject, which is where provenance lives. Four fields matter on every item you return:

• provider and application - your names, so Noded knows where it came from.
• uri - a link back to the item in the vendor's own UI.
• externalId - the vendor's stable ID for the item. This is the item's identity in Noded; the next concept explains why.

People do not have their own sync. They arrive as PersonReference objects on the things they appear in - a transcription's attendees, an email's to and from - and Noded matches them into the graph by email.

{
  "blockType": "transcription",
  "provider": "skylark",
  "application": "calls",
  "externalId": "rec_8f2e91c4",
  "uri": "https://app.skylark.example.com/rec/8f2e91c4",
  "title": "Acme Robotics - renewal call",
  "dateStart": 1757005200000,
  "duration": 2700,
  "organizedBy": { "email": "mia@acmerobotics.com",
                   "friendlyName": "Mia Chen" },
  "attendees": [
    { "email": "mia@acmerobotics.com",
      "friendlyName": "Mia Chen" },
    { "email": "sam@yourcompany.com",
      "friendlyName": "Sam Ortiz" }
  ],
  "transcript": { "title": "Transcript",
                  "text": "Mia: Thanks for joining..." },
  "processingStage": "processed"
}

Concept: external IDs are identity

Your provider returns the same items again and again - that is what polling means. The platform does not blindly insert them. A reconciler compares each incoming item to what is already stored, using externalId (together with your provider and application) as the key:

• New externalId - the item is created.
• Known externalId, changed content - the stored item is updated in place.
• Known externalId, same content - nothing happens.

So the one rule: an item's externalId must never change between syncs. Use the vendor's own stable ID. Never build it from anything that can shift - a title, a date, a page position. If your IDs are not stable, every sync creates duplicates of everything.

Tags work with the same idea from the other direction. The sync.identifier handler and externalIdLinks connect a vendor identity ("account 001Xj000ABC in the CRM") to a Noded tag ("Acme Robotics"), which is how records and insights land on the right account page.

// CORRECT - the vendor's own ID, stable forever
externalId: meeting.uuid

// WRONG - changes when the meeting is renamed
externalId: `${meeting.topic}-${meeting.date}`

{
  "created": 3,
  "updated": 1,
  "unchanged": 34,
  "skipped": 0,
  "failedCount": 0
}

Concept: the sync lifecycle

Once a user connects, the platform takes over. For each connection, it registers one sync per capability your provider definition declares. Then, on repeat:

1. Schedule. Most sync types run every few minutes; insights run daily; link is not scheduled at all - it fires when a user pastes a matching URL.
2. Call. The platform posts to your endpoint with the connection's credential and a filter. For paginated types, the filter carries a date window and, after the first page, your cursor.
3. Reconcile. Your items are matched by externalId and created, updated, or left alone.
4. Continue or wait. If you returned hasMore: true with a new cursor, the platform calls again immediately. Otherwise it waits for the next scheduled run.

Failures feed a health state for each sync - healthy, degraded, failing, paused - with growing backoff between attempts. A CredentialsExpiredError short-circuits this: syncing stops and the user is asked to reconnect, and a successful reconnect resumes it.

The practical effect: your handlers should be fast, honest, and stateless. Answer with one page of data, report the truth in pagination and warnings, and let the platform drive.

calendar        every ~5 minutes
email           every ~5 minutes  (paginated)
transcription   every ~5 minutes  (paginated)
table           every ~5 minutes
identifier      every ~5 minutes
insight         every ~24 hours
link            on demand (pasted URL)

{
  "connectedProvider": {
    "connectionId": "7b1e5c2a-9f4d-4e8b-a1c3-2d5e8f0a1b4c",
    "provider": "skylark",
    "application": "calls",
    "credential": {
      "accessToken": "sky_at_9f8e7d...",
      "email": "mia@acmerobotics.com"
    }
  },
  "transcriptionFilter": {
    "dateStart": 1756400400000,
    "dateEnd": 1757005200000,
    "limit": 30
  }
}

Pagination

Email and transcription syncs are paginated, and pagination is the part of the contract most worth learning properly. The platform drives the loop; you supply two signals per page:

• pagination.hasMore - is there another page?
• pagination.cursor - an opaque string that means "resume here". On the first call, filter.cursor is undefined. On every later call, it is exactly the string you returned last time. The platform never looks inside it - the format is entirely yours.

When you return hasMore: true and a cursor different from the one you received, the platform calls you again immediately - a large backfill completes in minutes, not days. When you return hasMore: false, or the cursor did not change, the loop ends and normal scheduling resumes.

Three cursor strategies cover every vendor API:

1. Pass-through. The vendor API has its own page token. Return it as your cursor; feed it back to the vendor next call. Most APIs work this way - the example on the right is real.

2. Structured. The vendor has no cursor, so you build one: a JSON string recording your position, such as a time window boundary. Advance it each page.

3. Offset. The vendor pages by number. The cursor is the offset as a string: "150".

Two more rules keep pages consistent:

Use the date window you are given. The platform computes dateStart / dateEnd once, on page one, and repeats the same window on every later page. Never call Date.now() to rebuild it - a shifting window skips or duplicates data at the seams.

Empty pages are fine. A time-window cursor may cross a quiet week: zero items, but the cursor advances. Return items: [] with hasMore: true and the new cursor. That is a normal page, not an error.

interface StandardSyncResponse<T> {
  items: T[];               // this page, may be empty
  metadata: {
    warnings?: string[];    // non-fatal notes
  };
  pagination: {
    hasMore: boolean;
    cursor?: string;        // omit on the last page
    pageSize: number;       // items actually returned
  };
}

async function syncTranscriptions(logger, credential,
    filter) {
  // First page: filter.cursor is undefined.
  // Later pages: it is the token we returned below.
  const page = await skylark.listRecordings(
    credential.accessToken, {
      from: filter.dateStart,   // given - never Date.now()
      to: filter.dateEnd,
      pageSize: filter.limit,
      pageToken: filter.cursor,
    });

  const items = page.recordings.map(toTranscription);

  return {
    items,
    metadata: {},
    pagination: {
      cursor: page.nextPageToken || undefined,
      hasMore: !!page.nextPageToken,
      pageSize: items.length,
    },
  };
}

{
  "items": [],
  "metadata": {},
  "pagination": {
    "hasMore": true,
    "cursor": "{\"resumeFrom\":1756832400000}",
    "pageSize": 0
  }
}

Conventions

A few rules hold across every handler. Learn them once and the reference below will feel repetitive - which is the point.

Timestamps are epoch milliseconds. Every date - dateStart, dateEnd, dateExpiry - is a number like 1757005200000. In JavaScript, Date.now() and new Date(value) speak this format natively. Convert at the vendor boundary and nowhere else.

The credential arrives in every body. Handlers receive it as their second argument, already validated by the SDK. The usual first line of a handler is const client = await authenticate(credential) - a helper of yours that builds a vendor client and throws CredentialsExpiredError if the token is dead.

instanceUrl is required for tag and upsert. These handlers reach into a specific vendor org, so the SDK rejects their requests up front when the credential has no instanceUrl. Capture it in auth.callback for vendors with per-tenant hosts.

Guard the application. Every connect and sync tells you which application it is for. Throw on names you do not serve - it turns a misconfiguration into a clear error instead of confusing data.

pageSize is the count you returned. Not the limit you were asked for. It feeds sync metrics.

Keep vendor code in one file per primitive. The convention across every real provider: src/index.ts holds only the startServer wiring, src/system/authentication.ts owns the vendor's auth, and each primitive gets its own file under src/primitives/. Handlers stay three lines: guard, authenticate, delegate.

bigfoot-skylark/
  src/
    index.ts                  # startServer wiring only
    system/
      authentication.ts       # connect/callback/refresh
    primitives/
      transcription.ts        # one file per primitive
  public/
    images/logo.svg           # served for free
  .env
  Dockerfile

async transcriptions(logger, credential, filter) {
  const client = await authenticate(credential);
  return await syncTranscriptions(logger, client, filter);
}

A complete example

Here is an entire working provider for a fictional meeting tool, Skylark. It supports OAuth sign-in and transcription sync - the same shape as the real Zoom provider, whose index.ts is 65 lines. Three files:

src/index.ts (right, top) is nothing but wiring. Every handler is a guard, an authenticate, and a delegate. This file should stay boring.

src/system/authentication.ts (right, middle) owns the vendor's OAuth. Notice what the callback stores: tokens, expiry, the user's email, and a display name in profile. That credential is the last state this service will ever hold - and it holds it only long enough to return it.

src/primitives/transcription.ts (right, bottom) does the real work: call the vendor inside the given window, map each recording to a Transcription with a stable externalId, and pass the vendor's page token through as the cursor.

That is the whole pattern. A tables provider swaps the primitive file for metadata/table handlers; an email provider swaps in emails. The wiring and auth files barely change.

import { startServer } from '@bigfootai/sdk';
import { syncTranscriptions }
  from './primitives/transcription.js';
import { authCallback, authenticate,
  getAuthenticationUrl, refreshToken }
  from './system/authentication.js';

await startServer({
  auth: {
    async connect(logger, application, connectionId) {
      if (application !== 'calls') {
        throw new Error('Unsupported application');
      }
      return {
        type: 'redirect',
        url: await getAuthenticationUrl(
          `${connectionId}.${application}`),
      };
    },
    async callback(logger, uri, queryParams) {
      const state = queryParams.get('state');
      if (!state) {
        throw new Error('Missing OAuth state');
      }
      return await authCallback(
        uri, state.split('.')[0], state);
    },
    async check(logger, application, credential) {
      await authenticate(credential);
      return { status: 'ok' };
    },
    async refresh(logger, connectedProvider) {
      return await refreshToken(connectedProvider);
    },
  },
  sync: {
    async transcriptions(logger, credential, filter) {
      const conn = await authenticate(credential);
      return await syncTranscriptions(
        logger, conn, filter);
    },
  },
});

export const getAuthenticationUrl = async (state) =>
  client.buildAuthorizationUrl(oauthConfig, {
    redirect_uri:
      `${process.env.BASE_URL}/auth/callback`,
    state,
    code_challenge: await pkceChallenge(),
    code_challenge_method: 'S256',
  }).toString();

export const authCallback = async
    (uri, connectionId, state) => {
  const tokens = await client.authorizationCodeGrant(
    oauthConfig, new URL(uri),
    { expectedState: state,
      pkceCodeVerifier: verifier });
  const me = await skylark.getMe(tokens.access_token);
  return {
    connectionId,
    credential: {
      accessToken: tokens.access_token,
      refreshToken: tokens.refresh_token,
      dateExpiry:
        Date.now() + tokens.expires_in * 1000,
      email: me.email,
      profile: me.display_name,
    },
  };
};

export const authenticate = async (credential) => {
  // Probe the vendor; throws on a dead token.
  await skylark.getMe(credential.accessToken);
  return credential;
};

export const syncTranscriptions = async
    (logger, credential, filter) => {
  const page = await skylark.listRecordings(
    credential.accessToken, {
      from: filter.dateStart,
      to: filter.dateEnd,
      pageSize: filter.limit,
      pageToken: filter.cursor,
    });

  const items = page.recordings.map((rec) => {
    const t = new Transcription(
      null, [], 'skylark', 'calls',
      rec.shareUrl, rec.uuid,        // uri, externalId
      ProcessingStage.Processed);
    t.title = rec.topic;
    t.dateStart = rec.startTime;
    t.duration = rec.durationSeconds;
    t.attendees = rec.participants.map((p) => ({
      email: p.email,
      friendlyName: p.name,
    }));
    t.transcript = { title: 'Transcript',
                     text: rec.transcriptText };
    return t;
  });

  return {
    items,
    metadata: {},
    pagination: {
      cursor: page.nextPageToken || undefined,
      hasMore: !!page.nextPageToken,
      pageSize: items.length,
    },
  };
};

Going live

A provider ships like any small web service. The checklist:

1. Test locally. Run pnpm dev, hit the health check, then exercise a sync handler directly with curl - the SDK validates the body exactly as the platform would, so a hand-built request is a faithful test (the panel on the right shows one).
2. Deploy. Build the template's Dockerfile and run it anywhere that serves HTTPS. Set the three environment variables. The template also includes a GitHub Actions workflow if you want CI to build and push the image.
3. Register. Submit your provider definition with instanceUrl pointing at your deployment. From this moment the platform knows how to reach you.
4. Connect and watch. Connect your own account from Noded's integrations page. Within a few minutes the first syncs fire. Watch your logs: you should see the connect round trip, then a steady rhythm of sync calls.

Before you call it done, walk the failure paths once on purpose: revoke your vendor token and confirm a CredentialsExpiredError shows up (and that reconnecting heals it), and run a big backfill to confirm the cursor keeps moving page after page.

curl -s http://localhost:5000/sync/transcription \
  -H "Authorization: Bearer $BIGFOOT_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "connectedProvider": {
      "connectionId": "local-test",
      "provider": "skylark",
      "application": "calls",
      "credential": {
        "accessToken": "sky_at_9f8e7d...",
        "email": "you@yourcompany.com"
      }
    },
    "transcriptionFilter": {
      "dateStart": 1756400400000,
      "dateEnd": 1757005200000,
      "limit": 5
    }
  }'

FROM node:24-alpine
WORKDIR /app
COPY . .
RUN corepack enable && pnpm install && pnpm build
EXPOSE 5000
CMD ["node", "dist/index.js"]