How to evaluate AI app quality with Braintrust and Webflow Cloud

An AI feature can return a 200, render without errors, and still be wrong. The model hands back a confident, incorrect answer that no exception tracker catches, because nothing crashed, so the regression sits in production until someone reads the output closely enough to notice.

The problem was that I had no way to measure output quality, no record of what real users were asking, and no test suite to catch a regression when I changed the prompt. Braintrust helps you to fix all three. It logs every AI call from your Webflow Cloud app, scores outputs against criteria you define, and runs an eval suite in CI so a prompt change that drops quality fails the build instead of reaching users.

The Braintrust SDK works in Webflow Cloud's Cloudflare Workers environment, but you must flush logs before the Route Handler returns its response. Cloudflare Workers can freeze or terminate a request context the moment the response is sent, and any unflushed log is lost. The fix is one await logger.flush() call, covered in Step 2.

What do you need to evaluate AI app quality with Braintrust in Webflow Cloud?

You need a Webflow Cloud app that already makes AI calls, a Braintrust account, and an API key from your chosen model provider. Braintrust has a free tier that covers development and low-volume production logging.

Here’s the full list of what you need:

• A Webflow Cloud project with at least one AI Route Handler (for example, the Gemini or OpenAI endpoint you already shipped)
• A Braintrust account (braintrust.dev/signup)
• A Braintrust API key (from Settings > API keys)
• A model provider API key (OpenAI, Anthropic, Gemini, whichever your app uses)
• Node.js 22.13.0 or higher locally for running evals from the command line. The Braintrust SDK sets no minimum, so the Webflow CLI 2.x floor is the binding constraint.

The split to keep in mind: logging runs in production on the edge; evals run locally or in CI on Node. The same Braintrust data structure backs both, which is what makes the workflow tight.

6 steps to evaluate AI app quality with Braintrust in Webflow Cloud

The workflow has two halves that feed each other. The runtime half adds Braintrust logging to your Webflow Cloud Route Handlers so every production AI call becomes a searchable trace. The development half writes an eval suite with test cases and scorers, runs it with the Braintrust CLI, and compares experiments to prove a change helped before you ship it. Production traces then become eval test cases, which closes the loop.

I run both halves on every AI feature now. Logging tells me what is happening in production. Evals tell me whether a change I am about to ship makes things better or worse.

1. Create a Braintrust project and get your API key

Sign up at braintrust.dev. On first login, Braintrust creates a default project; you can also create a named one from the dashboard home. Name it to match your Webflow Cloud app so logs and experiments stay grouped.

Go to Settings > API keys and create a key. Copy it.

Add the Braintrust key and your model provider key to .env.local and to Webflow Cloud's Settings > Environment Variables:

BRAINTRUST_API_KEY=sk-your-braintrust-key
OPENAI_API_KEY=sk-your-openai-key

Neither key gets a NEXT_PUBLIC_ prefix. Both are used server-side only, inside Route Handlers and eval scripts. A model provider key in the browser bundle lets anyone run inference on your account.

Then install the Braintrust SDK and the scoring library:

npm install braintrust autoevals

The braintrust package handles logging and evals. autoevals is a library of pre-built scorers (exact match, factuality, embedding similarity, and others) you'll use in Step 4.

2. Add Braintrust logging to a Webflow Cloud AI Route Handler

Braintrust logging captures every AI call as a trace you can inspect, filter, and later turn into test cases. The setup is initLogger plus wrapOpenAI around your model client so calls are traced automatically.

On Webflow Cloud, both have to be created inside the handler at request time because environment variables are not available at module load. The other edge-runtime detail is the flush call before the response returns.

Here’s the snippet:

// app/api/ai/route.ts
import { initLogger, wrapOpenAI } from 'braintrust'
import OpenAI from 'openai'
import { NextResponse, type NextRequest } from 'next/server'

export async function POST(request: NextRequest) {
  // Initialize at request time. On Webflow Cloud, environment variables
  // are available at runtime only, not at module load, so the logger and
  // client must be created inside the handler.
  const logger = initLogger({
    projectName: 'my-webflow-cloud-app',
    apiKey: process.env.BRAINTRUST_API_KEY,
  })

  // wrapOpenAI traces every model call automatically.
  const client = wrapOpenAI(
    new OpenAI({ apiKey: process.env.OPENAI_API_KEY })
  )

  const { prompt } = await request.json() as { prompt: string }

  try {
    const completion = await client.chat.completions.create({
      model: 'gpt-4o',
      messages: [
        { role: 'system', content: 'You are a helpful assistant.' },
        { role: 'user', content: prompt },
      ],
    })

    const output = completion.choices[0].message.content

    // CRITICAL on edge: flush before the response returns.
    // Cloudflare Workers can freeze the context after the response,
    // dropping any unflushed logs.
    await logger.flush()

    return NextResponse.json({ output })
  } catch (error) {
    await logger.flush()
    return NextResponse.json({ error: 'AI request failed' }, { status: 500 })
  }
}

The await logger.flush() call is the one line people miss. Without it, logging works perfectly in local development (where the Node process stays alive) and then silently drops most logs in production (where the Worker freezes after responding). Flush in both the success and error paths so failed calls get logged too.

3. Trace richer context in your Webflow Cloud AI calls

wrapOpenAI traces the model call itself. To make logs useful for evaluation, attach the metadata you'll want to filter and score later: the user's input, your app's final output, and any context, such as a user ID or feature name. Use traced to wrap a unit of work in a span and span.log to attach fields.

Here’s what it looks like:

// app/api/ai/route.ts (excerpt)
import { initLogger, wrapOpenAI, traced } from 'braintrust'
import OpenAI from 'openai'

export async function POST(request: NextRequest) {
  // Create the logger and client at request time (see Step 2).
  const logger = initLogger({
    projectName: 'my-webflow-cloud-app',
    apiKey: process.env.BRAINTRUST_API_KEY,
  })
  const client = wrapOpenAI(
    new OpenAI({ apiKey: process.env.OPENAI_API_KEY })
  )

  const { prompt, userId } = await request.json() as {
    prompt: string
    userId?: string
  }

  const output = await traced(
    async (span) => {
      const completion = await client.chat.completions.create({
        model: 'gpt-4o',
        messages: [
          { role: 'system', content: 'You are a helpful assistant.' },
          { role: 'user', content: prompt },
        ],
      })

      const result = completion.choices[0].message.content

      // Attach structured fields for later filtering and scoring.
      span.log({
        input: prompt,
        output: result,
        metadata: { userId, feature: 'content-assistant' },
      })

      return result
    },
    { name: 'content-assistant' }
  )

  await logger.flush()
  return NextResponse.json({ output })
}

The metadata fields become filterable columns in the Braintrust Logs view. When a specific feature starts producing bad output, you filter to feature:content-assistant and read the actual prompts and responses instead of guessing. The input and output fields are what you'll later use to create an eval dataset.

If your Webflow Cloud app routes model calls through Cloudflare AI Gateway, wrapOpenAI traces those too. Point the OpenAI client baseURL at your gateway endpoint, and Braintrust captures the calls the same way, with the gateway's caching and rate limiting in front.

4. Write a Braintrust eval suite for your Webflow Cloud AI feature

An eval measures AI quality against test cases instead of vibes. Every eval has three parts: data (test cases with inputs and expected outputs), a task (the AI function under test), and scores (functions that grade the output).

Create an eval file alongside your app code; it runs on Node, not on the edge, so it lives outside your Route Handlers.

// evals/content-assistant.eval.ts
import { Eval } from 'braintrust'
import { Factuality, ExactMatch } from 'autoevals'
import OpenAI from 'openai'

const client = new OpenAI({ apiKey: process.env.OPENAI_API_KEY })

// The same logic your Route Handler runs, isolated for testing.
async function runAssistant(prompt: string): Promise<string> {
  const completion = await client.chat.completions.create({
    model: 'gpt-4o',
    messages: [
      { role: 'system', content: 'You are a helpful assistant.' },
      { role: 'user', content: prompt },
    ],
  })
  return completion.choices[0].message.content ?? ''
}

Eval('my-webflow-cloud-app', {
  experimentName: 'content-assistant-v1',
  // Data: test cases with inputs and expected outputs.
  data: [
    {
      input: 'Summarize the benefits of server-side rendering in one sentence.',
      expected:
        'Server-side rendering improves initial load time and SEO by sending fully rendered HTML to the browser.',
    },
    {
      input: 'What runtime does a Next.js app run on in Webflow Cloud?',
      expected: 'Cloudflare Workers, via the OpenNext Cloudflare adapter.',
    },
  ],
  // Task: the function being evaluated.
  task: async (input) => runAssistant(input),
  // Scores: how output quality is measured.
  // Factuality is an LLM-as-a-judge scorer; it grades whether
  // the output is consistent with the expected answer.
  scores: [Factuality],
})

Factuality is an LLM-as-a-judge scorer from autoevals; it grades whether the output agrees with the expected answer rather than requiring a character-for-character match. For outputs with one correct value (a slug, a yes/no, an ID), use ExactMatch instead. You can also write a custom scorer: any function that returns a score between 0 and 1.

5. Run the Braintrust eval and compare experiments

Running the eval creates an experiment, a permanent record of how your AI performed on the test cases. Comparing two experiments is how you prove a prompt or model change actually helped before it ships to your Webflow Cloud app.

Install the Braintrust CLI and run the eval:

# Install the bt CLI (macOS and Linux)
curl -fsSL https://bt.dev/cli/install.sh | bash

# Run the eval
bt eval evals/content-assistant.eval.ts

The terminal prints a summary table and a link to the experiment in the Braintrust UI. To test a change, edit the system prompt or swap the model, bump the experimentName (for example, content-assistant-v2), and run again. Braintrust shows the two experiments side by side, with the score delta for each test case.

This is the moment the whole setup pays off. Instead of "the new prompt feels better," you get a per-case score delta: which cases improved, which regressed, and by how much. A change that drops the score is visible immediately, before any user sees it.

6. Turn Webflow Cloud production logs into Braintrust eval datasets

The strongest test cases come from real usage, not your imagination. Because production logs and eval datasets share the same structure in Braintrust, promoting a logged trace into a test case takes a few clicks. This is how the runtime and development halves connect.

In the Braintrust UI, open the Logs page, filter to the feature or failure pattern you care about (using the metadata you attached in Step 3), select the traces that represent cases you want to guard against, and add them to a dataset.

Then, point your eval at that dataset instead of the hard-coded array:

import { Eval, initDataset } from 'braintrust'

Eval('my-webflow-cloud-app', {
  experimentName: 'content-assistant-v3',
  // Pull test cases from a dataset built from production logs.
  data: initDataset({ project: 'my-webflow-cloud-app', dataset: 'production-failures' }),
  task: async (input) => runAssistant(input),
  scores: [Factuality],
})

Every time a user hits a bad output in production, you capture it, add it to the dataset, and your eval suite grows to cover it. A regression that ships once never ships twice. To make this automatic, run bt eval in your CI pipeline on every pull request; a change that drops the score below your threshold fails the build.

What breaks AI quality evaluation in Webflow Cloud with Braintrust?

Most Braintrust problems on Webflow Cloud fall into four categories: logs that appear locally but vanish in production, eval files that refuse to run, factuality scores that come back low even for good output, and experiments that never show up in the dashboard. Each one traces back to a specific cause in either the edge runtime or the eval setup, and each has a direct fix.

The four below cover what I hit most often, in the order they tend to surface.

Logs work locally but are missing in production

This is the flush problem from Step 2. In local development, the Node process stays alive long enough for Braintrust's background flush to complete, so logs appear. On Webflow Cloud's edge runtime, the Worker context can be frozen the instant the response returns, dropping anything not yet sent.

Add await logger.flush() before every return in the Route Handler, including error paths.

The eval fails to run with a module or import error

The eval file runs on Node via bt eval, not in the Worker. Keep eval files out of the app/ directory so Next.js doesn't try to bundle them as routes, and do not add a runtime export to them. Webflow Cloud runs your app through the OpenNext Cloudflare adapter, which does not support Next.js's edge runtime anywhere in the project.

An evals/ folder at the project root is the convention. If imports still fail, confirm braintrust and autoevals are installed and that your BRAINTRUST_API_KEY is set in the shell running the eval.

All Factuality scores come back low, even for good outputs

The Factuality scorer compares output against the expected field. If your expected answers are phrased very differently from how the model responds (different length, different structure), the judge may grade them as inconsistent.

Either loosen the expected answers to capture the core fact rather than exact phrasing, or switch to a scorer better matched to your task. For open-ended generation, a custom LLM-as-a-judge scorer with a rubric specific to your use case beats a generic one.

Experiments don't appear in the Braintrust UI

Check the terminal output of bt eval for the experiment link and any error lines. The most common cause is a missing or wrong BRAINTRUST_API_KEY in the environment running the eval.

Confirm that the project name in your eval file matches the project you're viewing in the UI; a typo can create a separate project that looks empty.

Extend AI quality evaluation across your Webflow Cloud app

The setup in this guide gives you the core loop: log production AI calls, write evals, compare experiments, and feed real failures back into the test suite. Both halves extend cleanly.

For logging, add online scoring so production traces get graded automatically as they arrive, not just in offline evals. Braintrust can run a scorer against a sample of live logs and chart the score over time, turning quality into a dashboard metric you monitor like latency or error rate.

For evals, wire bt eval into your CI pipeline as a required check. Set a minimum score threshold per scorer; a pull request that drops factuality below the bar fails before merge. This is the difference between hoping a prompt change is safe and knowing it.

Explore Webflow + Braintrust to see the other patterns this connects to, from CMS-synced evaluation dashboards to quality gates that hold a publish until scores pass.