How I modernized a legacy form system with integrations and reporting.

A form system that only accepts fields and stores rows is becoming less useful by the day. Modern teams expect a submission to update analytics, create a contact, record consent, notify somebody, and produce reports without requiring a new deployment for every form.

I ran into that problem while taking over an older form platform. Adding another one-off callback to every form would have made the system harder to trust and almost impossible to scale. I needed the new offering to be more robust than the system it replaced, so I started by separating form collection from everything that happens after submission.

The integration layer

In plain terms, the integration layer is configuration that can be built once and attached to any form. The form schema contains a list of supported integration instances. Each instance declares a known type, its required configuration, field mappings, and optional conditions.

I deliberately avoided a generic “send this payload to any URL” feature. Arbitrary endpoints turn the platform into an uncontrolled proxy: they can leak submission data, reach internal services, produce unpredictable responses, and make authentication, retries, and support impossible to standardize. An allowlisted adapter registry provides extension without giving up control.

export const integrationRegistry = {
  analyticsEvent: sendAnalyticsEvent,
  hubspotContact: syncHubspotContact,
  smsSubscription: subscribeSmsContact,
  internalNotification: sendInternalNotification,
} as const

export type IntegrationType = keyof typeof integrationRegistry

Give every integration a contract

Each adapter needs different information. An analytics event needs an event name. A CRM contact needs field mappings. An SMS subscription needs a phone field and explicit consent evidence. A discriminated schema keeps those requirements close to the integration type instead of spreading conditional checks across the executor.

import {z} from 'zod'

const fieldMap = z.record(z.string(), z.string().min(1))

export const integrationSchema = z.discriminatedUnion('type', [
  z.object({
    id: z.string().uuid(),
    type: z.literal('analyticsEvent'),
    eventName: z.string().min(1).max(80),
    fieldMappings: fieldMap.default({}),
    enabled: z.boolean().default(true),
  }),
  z.object({
    id: z.string().uuid(),
    type: z.literal('hubspotContact'),
    connectionId: z.string().uuid(),
    fieldMappings: fieldMap,
    enabled: z.boolean().default(true),
  }),
  z.object({
    id: z.string().uuid(),
    type: z.literal('smsSubscription'),
    connectionId: z.string().uuid(),
    phoneField: z.string().min(1),
    consentField: z.string().min(1),
    listId: z.string().min(1),
    enabled: z.boolean().default(true),
  }),
])

The configuration stores a connection reference, never the provider secret itself. Credentials remain server-side in a secrets store. Field mappings allow one adapter to work with forms that call the same value email, contact_email, or workEmail without hard-coding those names into provider code.

Harden the boundary with validation

Static TypeScript types disappear at runtime, while form configuration, stored submissions, cron invocations, and provider responses all arrive as untrusted data. I validate at every boundary: when an administrator saves a form, when a visitor submits it, immediately before an event is executed, and when an external API responds.

const parsed = integrationSchema.safeParse(rawIntegration)

if (!parsed.success) {
  return {
    ok: false,
    code: 'INVALID_INTEGRATION_CONFIG',
    issues: parsed.error.issues.map(({path, message}) => ({path, message})),
  }
}

const integration = parsed.data

The save-time validator should also verify that every mapped field exists on the selected form, required consent fields are boolean or equivalent, conditions reference valid fields, and the connection belongs to the same account. Rejecting an impossible configuration before publication is cheaper than discovering it during a live submission.

Persist the submission before calling providers

The visitor should not wait while several external APIs respond. The request validates the submission, saves it, creates durable integration events, and returns success. Workers process those events independently. A temporary HubSpot or messaging outage must not make the visitor believe the form was lost.

const submission = await db.transaction(async (tx) => {
  const saved = await tx.submission.create({
    data: {formId, payload: validatedSubmission},
  })

  await tx.integrationEvent.createMany({
    data: activeIntegrations.map((integration) => ({
      submissionId: saved.id,
      integrationId: integration.id,
      integrationType: integration.type,
      status: 'pending',
      attemptCount: 0,
      nextAttemptAt: new Date(),
      idempotencyKey: `${saved.id}:${integration.id}`,
      configVersion: form.version,
    })),
  })

  return saved
})

return Response.json({submissionId: submission.id}, {status: 202})

Conditional behavior belongs in configuration

Once integrations are data rather than embedded callbacks, conditions become possible. An SMS adapter can run only when the visitor explicitly selects SMS and checks the consent field. A sales notification can run only for qualified leads. Keep the operator set small and typed rather than evaluating arbitrary JavaScript.

{
  "when": {
    "field": "contactPreference",
    "operator": "equals",
    "value": "sms"
  },
  "integrationId": "sms-opt-in"
}

The reporting layer

The second capability I added was scheduled reporting. A form can declare report recipients, an email subject, and a frequency. A Vercel Cron Job invokes one secured route, and that route finds reports that are due, gathers submissions since each report’s last successful cutoff, and emails a bounded summary or attachment.

export const reportSchema = z.object({
  enabled: z.boolean(),
  recipients: z.array(z.string().email()).min(1).max(20),
  subject: z.string().min(1).max(140),
  frequency: z.enum(['daily', 'weekly', 'monthly']),
  timezone: z.string().min(1),
  lastCompletedAt: z.coerce.date().nullable(),
})

The cutoff is as important as the schedule. A report should cover a stored interval such as [lastCompletedAt, reportUntil), then advance lastCompletedAt only after the email provider confirms success. That makes a failed run replayable without silently skipping leads.

{
  "$schema": "https://openapi.vercel.sh/vercel.json",
  "crons": [
    {
      "path": "/api/cron/form-reports",
      "schedule": "0 * * * *"
    },
    {
      "path": "/api/cron/reconcile-integrations",
      "schedule": "*/10 * * * *"
    }
  ]
}

Vercel schedules use UTC, so the route calculates which reports are due in their configured time zones. I also validate the Authorization header against CRON_SECRET. A cron route is still a production endpoint and should not be triggerable by any visitor who discovers its URL.

export async function GET(request: Request) {
  if (request.headers.get('authorization') !==
      `Bearer ${process.env.CRON_SECRET}`) {
    return new Response('Unauthorized', {status: 401})
  }

  const dueReports = await findDueReports(new Date())
  const results = await processInBoundedBatches(dueReports, 10, sendReport)

  return Response.json({processed: results.length})
}

Avoid loading every lead for every form into one function invocation. Page through due reports, bound concurrency, stream or generate manageable attachments, and respect the function duration. The reporting job should record its own run, interval, recipient set, provider message ID, result, and failure reason.

Make failures explainable with Sentry

Retries are useful only when you can tell why an event failed. I added Sentry around the boundaries where errors have meaning: configuration validation, condition evaluation, field mapping, provider authentication, provider requests, report generation, email delivery, and reconciliation.

A stack trace alone is not enough. The event needs identifiers that let an operator find the durable record and replay it: form ID, submission ID, integration event ID, integration type, configuration version, attempt number, provider request ID, and deployment environment.

import * as Sentry from '@sentry/nextjs'

try {
  await adapter.execute({submission, integration, idempotencyKey})
  await markCompleted(event.id)
} catch (error) {
  Sentry.withScope((scope) => {
    scope.setTag('integration.type', event.integrationType)
    scope.setTag('integration.status', 'failed')
    scope.setContext('integration_event', {
      eventId: event.id,
      formId: submission.formId,
      submissionId: submission.id,
      integrationId: event.integrationId,
      configVersion: event.configVersion,
      attemptCount: event.attemptCount + 1,
      providerRequestId: getProviderRequestId(error),
    })
    Sentry.captureException(error)
  })

  await markFailedWithBackoff(event.id, error)
}

Build replay into the event model

A failed event should contain enough durable information to run again after the underlying problem is fixed. I store a reference to the immutable submission, the integration ID and type, the configuration version used, an idempotency key, attempt timestamps, a normalized error code, and the next eligible attempt time.

Versioning matters because an administrator may edit a form after the failure. Replaying against today’s mapping can send different data to a different destination. Either retain an immutable configuration snapshot with secrets excluded, or store a configuration version that the executor can resolve.

type IntegrationEventStatus =
  | 'pending'
  | 'processing'
  | 'retryable'
  | 'completed'
  | 'dead'

type IntegrationEvent = {
  id: string
  submissionId: string
  integrationId: string
  configVersion: number
  idempotencyKey: string
  status: IntegrationEventStatus
  attemptCount: number
  nextAttemptAt: Date
  lastErrorCode?: string
  lastProviderRequestId?: string
}

Use a reconciler for work that fell through the cracks

The normal executor handles the happy path and immediate failures. A separate reconciliation cron is the safety net. It looks for pending events that were never picked up, retryable events whose backoff has elapsed, and processing events whose lease expired after a timeout or deployment interruption.

const candidates = await db.integrationEvent.findMany({
  where: {
    OR: [
      {status: 'pending', nextAttemptAt: {lte: now}},
      {status: 'retryable', nextAttemptAt: {lte: now}},
      {status: 'processing', leaseExpiresAt: {lt: now}},
    ],
    attemptCount: {lt: MAX_ATTEMPTS},
  },
  orderBy: {nextAttemptAt: 'asc'},
  take: 100,
})

for (const event of candidates) {
  if (await acquireLease(event.id)) {
    await executeIntegrationEvent(event.id)
  }
}

Acquiring a lease must be atomic so overlapping cron invocations do not process the same row concurrently. Exponential backoff prevents an unhealthy provider from being hammered. After the maximum attempt count, move the event to dead, alert with context, and require a deliberate replay after the cause is understood.

This reconciler is necessary on Vercel because a failed cron invocation is not automatically retried, cron events can occasionally be delivered more than once, and a new run can overlap a slow previous run. Locks and idempotency address different failure modes; a reliable system needs both.

The architecture after the refactor

Browser
  -> validate submission
  -> store submission + integration events atomically
  -> return success

Executor
  -> claim event
  -> validate versioned configuration
  -> evaluate conditions and map fields
  -> call allowlisted adapter with idempotency key
  -> complete or schedule retry

Scheduled jobs
  -> send due form reports
  -> reconcile abandoned and retryable events

Observability
  -> Sentry IDs and sanitized context
  -> provider request IDs
  -> durable event and report-run history

What made the system trustworthy

The useful part was not adding one analytics call or one CRM connector. It was creating a controlled extension point. New integrations now enter through a registry, declare a schema, reuse the same event lifecycle, emit the same operational context, and inherit retry and reconciliation behavior.

The reporting layer follows the same principle. Schedules are configuration, but execution is centralized, secured, observable, bounded, and replayable. Schema validation stops impossible setups early. Sentry tells us which durable record failed without copying sensitive payloads. Reconciliation turns transient provider and serverless failures into recoverable states instead of missing leads.