Braintrust TypeScript SDK

This page provides a complete, auto-generated reference for the Braintrust TypeScript SDK. For usage guidance on important APIs, see the API Reference. Also see the Braintrust TypeScript SDK on GitHub.

Installation

npm install braintrust

pnpm add braintrust

Starting with v2.0.0, if you’re using the Vercel AI SDK integration or other features that require schema validation, you must install zod as a peer dependency: npm install zod

Functions

BaseExperiment

Use this to specify that the dataset should actually be the data from a previous (base) experiment. If you do not specify a name, Braintrust will automatically figure out the best base experiment to use based on your git history (or fall back to timestamps).

Object

string

The name of the base experiment to use. If unspecified, Braintrust will automatically figure out the best base using your git history (or fall back to timestamps).

BraintrustMiddleware

Creates a Braintrust middleware for AI SDK v2 that automatically traces generateText and streamText calls with comprehensive metadata and metrics.

MiddlewareConfig

Configuration options for the middleware

buildLocalSummary

buildLocalSummary function

EvaluatorDef

string

required

string

required

string

An optional experiment id to use as a base. If specified, the new experiment will be summarized and compared to this experiment. This takes precedence over baseExperimentName if specified.

string

An optional experiment name to use as a base. If specified, the new experiment will be summarized and compared to this experiment.

EvalData

required

A function that returns a list of inputs, expected outputs, and metadata.

string

An optional description for the experiment.

ErrorScoreHandler

Optionally supply a custom function to specifically handle score values when tasks or scoring functions have errored. A default implementation is exported as defaultErrorScoreHandler which will log a 0 score to the root span for any scorer that was not run.

string

An optional name for the experiment.

Object

Optional settings for collecting git metadata. By default, will collect all git metadata fields allowed in org-level settings.

boolean

Whether the experiment should be public. Defaults to false.

number

The maximum number of tasks/scorers that will be run concurrently. Defaults to undefined, in which case there is no max concurrency.

Record

Optional additional metadata for the experiment.

Parameters

A set of parameters that will be passed to the evaluator. Can contain array values that will be converted to single values in the task.

string

If specified, uses the given project ID instead of the evaluator’s name to identify the project.

null | Object

Optionally explicitly specify the git metadata for this experiment. This takes precedence over gitMetadataSettings if specified.

EvalScorer[]

required

A set of functions that take an input, output, and expected value and return a score.

AbortSignal

An abort signal that can be used to stop the evaluation.

BraintrustState

If specified, uses the logger state to initialize Braintrust objects. If unspecified, falls back to the global state (initialized using your API key).

boolean

Whether to summarize the scores of the experiment after it has run. Defaults to true.

EvalTask

required

A function that takes an input and returns an output.

number

The duration, in milliseconds, after which to time out the evaluation. Defaults to undefined, in which case there is no timeout.

number

The number of times to run the evaluator per input. This is useful for evaluating applications that have non-deterministic behavior and gives you both a stronger aggregate measure and a sense of the variance in the results.

boolean

Whether to update an existing experiment with experiment_name if one exists. Defaults to false.

EvalResult[]

ScoreAccumulator

createFinalValuePassThroughStream

Create a stream that passes through the final value of the stream. This is used to implement BraintrustStream.finalValue().

Object

A function to call with the final value of the stream.

Object

currentExperiment

Returns the currently-active experiment (set by init). Returns undefined if no current experiment has been set.

OptionalStateArg

currentLogger

Returns the currently-active logger (set by initLogger). Returns undefined if no current logger has been set.

unknown

currentSpan

Return the currently-active span for logging (set by one of the traced methods). If there is no active span, returns a no-op span object, which supports the same interface as spans but does no logging. See Span for full details.

OptionalStateArg

defaultErrorScoreHandler

defaultErrorScoreHandler function

Object

EvalCase

required

Span

required

string[]

required

deserializePlainStringAsJSON

deserializePlainStringAsJSON function

string

devNullWritableStream

devNullWritableStream function

Eval

Eval function

string

Evaluator

string

An optional experiment id to use as a base. If specified, the new experiment will be summarized and compared to this experiment. This takes precedence over baseExperimentName if specified.

string

An optional experiment name to use as a base. If specified, the new experiment will be summarized and compared to this experiment.

EvalData

required

A function that returns a list of inputs, expected outputs, and metadata.

string

An optional description for the experiment.

ErrorScoreHandler

string

An optional name for the experiment.

Object

Optional settings for collecting git metadata. By default, will collect all git metadata fields allowed in org-level settings.

boolean

Whether the experiment should be public. Defaults to false.

number

The maximum number of tasks/scorers that will be run concurrently. Defaults to undefined, in which case there is no max concurrency.

Record

Optional additional metadata for the experiment.

Parameters

A set of parameters that will be passed to the evaluator. Can contain array values that will be converted to single values in the task.

string

If specified, uses the given project ID instead of the evaluator’s name to identify the project.

null | Object

Optionally explicitly specify the git metadata for this experiment. This takes precedence over gitMetadataSettings if specified.

EvalScorer[]

required

A set of functions that take an input, output, and expected value and return a score.

AbortSignal

An abort signal that can be used to stop the evaluation.

BraintrustState

If specified, uses the logger state to initialize Braintrust objects. If unspecified, falls back to the global state (initialized using your API key).

boolean

Whether to summarize the scores of the experiment after it has run. Defaults to true.

EvalTask

required

A function that takes an input and returns an output.

number

The duration, in milliseconds, after which to time out the evaluation. Defaults to undefined, in which case there is no timeout.

number

boolean

Whether to update an existing experiment with experiment_name if one exists. Defaults to false.

string | ReporterDef | EvalOptions

flush

Flush any pending rows to the server.

OptionalStateArg

getContextManager

getContextManager function

getIdGenerator

Factory function that creates a new ID generator instance each time. This eliminates global state and makes tests parallelizable. Each caller gets their own generator instance.

getPromptVersions

Get the versions for a prompt.

string

The ID of the project to query

string

The ID of the prompt to get versions for

getSpanParentObject

Mainly for internal use. Return the parent object for starting a span in a global context. Applies precedence: current span > propagated parent string > experiment > logger.

unknown

string

init

Readonly

Options for configuring init().

string

The API key to use. If the parameter is not specified, will try to use the BRAINTRUST_API_KEY environment variable.

string

The URL of the Braintrust App. Defaults to https://www.braintrust.dev. You should not need to change this unless you are doing the “Full” deployment.

boolean

If true, disables the local span cache used to optimize scorer access to trace data. When disabled, scorers will always fetch spans from the server. Defaults to false.

Object

A custom fetch implementation to use.

boolean

By default, the SDK installs an event handler that flushes pending writes on the beforeExit event. If true, this event handler will not be installed.

Object

Calls this function if there’s an error in the background flusher.

string

The name of a specific organization to connect to. Since API keys are scoped to organizations, this parameter is usually unnecessary unless you are logging in with a JWT.

string

AnyDataset | DatasetRef

string

GitMetadataSettings

boolean

Record

string

RepoInfo

boolean

BraintrustState

boolean

initDataset

Create a new dataset in a specified project. If the project does not exist, it will be created.

Readonly

Options for configuring initDataset().

string

The API key to use. If the parameter is not specified, will try to use the BRAINTRUST_API_KEY environment variable.

string

The URL of the Braintrust App. Defaults to https://www.braintrust.dev. You should not need to change this unless you are doing the “Full” deployment.

boolean

If true, disables the local span cache used to optimize scorer access to trace data. When disabled, scorers will always fetch spans from the server. Defaults to false.

Object

A custom fetch implementation to use.

boolean

By default, the SDK installs an event handler that flushes pending writes on the beforeExit event. If true, this event handler will not be installed.

Object

Calls this function if there’s an error in the background flusher.

string

The name of a specific organization to connect to. Since API keys are scoped to organizations, this parameter is usually unnecessary unless you are logging in with a JWT.

string

Record

string

BraintrustState

string

initExperiment

Alias for init(options).

Readonly

string

The API key to use. If the parameter is not specified, will try to use the BRAINTRUST_API_KEY environment variable.

string

The URL of the Braintrust App. Defaults to https://www.braintrust.dev. You should not need to change this unless you are doing the “Full” deployment.

boolean

If true, disables the local span cache used to optimize scorer access to trace data. When disabled, scorers will always fetch spans from the server. Defaults to false.

Object

A custom fetch implementation to use.

boolean

By default, the SDK installs an event handler that flushes pending writes on the beforeExit event. If true, this event handler will not be installed.

Object

Calls this function if there’s an error in the background flusher.

string

The name of a specific organization to connect to. Since API keys are scoped to organizations, this parameter is usually unnecessary unless you are logging in with a JWT.

string

AnyDataset | DatasetRef

string

GitMetadataSettings

boolean

Record

string

RepoInfo

boolean

BraintrustState

boolean

initFunction

Creates a function that can be used as a task or scorer in the Braintrust evaluation framework. The returned function wraps a Braintrust function and can be passed directly to Eval(). When used as a task:

const myFunction = initFunction({projectName: "myproject", slug: "myfunction"});
await Eval("test", {
  task: myFunction,
  data: testData,
  scores: [...]
});

When used as a scorer:

const myScorer = initFunction({projectName: "myproject", slug: "myscorer"});
await Eval("test", {
  task: someTask,
  data: testData,
  scores: [myScorer]
});

Object

Options for the function.

string

required

The project name containing the function.

string

required

The slug of the function to invoke.

BraintrustState

Optional Braintrust state to use.

string

Optional version of the function to use. Defaults to latest.

initLogger

Create a new logger in a specified project. If the project does not exist, it will be created.

Readonly

Additional options for configuring init().

string

The API key to use. If the parameter is not specified, will try to use the BRAINTRUST_API_KEY environment variable.

string

The URL of the Braintrust App. Defaults to https://www.braintrust.dev. You should not need to change this unless you are doing the “Full” deployment.

boolean

If true, disables the local span cache used to optimize scorer access to trace data. When disabled, scorers will always fetch spans from the server. Defaults to false.

Object

A custom fetch implementation to use.

boolean

By default, the SDK installs an event handler that flushes pending writes on the beforeExit event. If true, this event handler will not be installed.

Object

Calls this function if there’s an error in the background flusher.

string

The name of a specific organization to connect to. Since API keys are scoped to organizations, this parameter is usually unnecessary unless you are logging in with a JWT.

OrgProjectMetadata

string

boolean

BraintrustState

invoke

Invoke a Braintrust function, returning a BraintrustStream or the value as a plain Javascript object.

unknown

The arguments for the function (see InvokeFunctionArgs for more details).

The type of the global function to invoke. If unspecified, defaults to ‘scorer’ for backward compatibility.

string

The ID of the function to invoke.

string

The name of the global function to invoke.

Input

required

The input to the function. This will be logged as the input field in the span.

Additional OpenAI-style messages to add to the prompt (only works for llm functions).

Record

Additional metadata to add to the span. This will be logged as the metadata field in the span. It will also be available as the {{metadata}} field in the prompt and as the metadata argument to the function.

null | "text" | "auto" | "parallel" | "json"

The mode of the function. If “auto”, will return a string if the function returns a string, and a JSON object otherwise. If “parallel”, will return an array of JSON objects with one object per tool call.

string | Exportable

The parent of the function. This can be an existing span, logger, or experiment, or the output of .export() if you are distributed tracing. If unspecified, will use the same semantics as traced() to determine the parent and no-op if not in a tracing context.

string

The ID of the project to use for execution context (API keys, project defaults, etc.). This is not the project the function belongs to, but the project context for the invocation.

string

The name of the project containing the function to invoke.

string

The ID of the function in the prompt session to invoke.

string

The ID of the prompt session to invoke the function from.

unknown

A Zod schema to validate the output of the function and return a typed value. This is only used if stream is false.

string

The slug of the function to invoke.

BraintrustState

(Advanced) This parameter allows you to pass in a custom login state. This is useful for multi-tenant environments where you are running functions from different Braintrust organizations.

Stream

Whether to stream the function’s output. If true, the function will return a BraintrustStream, otherwise it will return the output of the function as a JSON object.

boolean

Whether to use strict mode for the function. If true, the function will throw an error if the variable names in the prompt do not match the input keys.

string[]

Tags to add to the span. This will be logged as the tags field in the span.

string

The version of the function to invoke.

string

The API key to use. If the parameter is not specified, will try to use the BRAINTRUST_API_KEY environment variable.

string

The URL of the Braintrust App. Defaults to https://www.braintrust.dev. You should not need to change this unless you are doing the “Full” deployment.

boolean

If true, disables the local span cache used to optimize scorer access to trace data. When disabled, scorers will always fetch spans from the server. Defaults to false.

Object

A custom fetch implementation to use.

boolean

By default, the SDK installs an event handler that flushes pending writes on the beforeExit event. If true, this event handler will not be installed.

Object

Calls this function if there’s an error in the background flusher.

string

The name of a specific organization to connect to. Since API keys are scoped to organizations, this parameter is usually unnecessary unless you are logging in with a JWT.

isTemplateFormat

isTemplateFormat function

unknown

loadPrompt

Load a prompt from the specified project.

LoadPromptOptions

Options for configuring loadPrompt().

string

The API key to use. If the parameter is not specified, will try to use the BRAINTRUST_API_KEY environment variable.

string

The URL of the Braintrust App. Defaults to https://www.braintrust.dev. You should not need to change this unless you are doing the “Full” deployment.

boolean

If true, disables the local span cache used to optimize scorer access to trace data. When disabled, scorers will always fetch spans from the server. Defaults to false.

Object

A custom fetch implementation to use.

boolean

By default, the SDK installs an event handler that flushes pending writes on the beforeExit event. If true, this event handler will not be installed.

Object

Calls this function if there’s an error in the background flusher.

string

The name of a specific organization to connect to. Since API keys are scoped to organizations, this parameter is usually unnecessary unless you are logging in with a JWT.

DefaultPromptArgs

string

boolean

string

BraintrustState

string

log

Log a single event to the current experiment. The event will be batched and uploaded behind the scenes.

ExperimentLogFullArgs

The event to log. See Experiment.log for full details.

unknown

required

string

required

logError

logError function

Span

string

required

Row ID of the span.

"span"

required

string

required

Root span ID of the span.

string

required

Span ID of the span.

string[]

required

Parent span IDs of the span.

unknown

Log into Braintrust. This will prompt you for your API token, which you can find at https://www.braintrust.dev/app/token. This method is called automatically by init().

unknown

Options for configuring login().

string

The API key to use. If the parameter is not specified, will try to use the BRAINTRUST_API_KEY environment variable.

string

The URL of the Braintrust App. Defaults to https://www.braintrust.dev. You should not need to change this unless you are doing the “Full” deployment.

boolean

If true, disables the local span cache used to optimize scorer access to trace data. When disabled, scorers will always fetch spans from the server. Defaults to false.

Object

A custom fetch implementation to use.

boolean

By default, the SDK installs an event handler that flushes pending writes on the beforeExit event. If true, this event handler will not be installed.

Object

Calls this function if there’s an error in the background flusher.

string

The name of a specific organization to connect to. Since API keys are scoped to organizations, this parameter is usually unnecessary unless you are logging in with a JWT.

Login again, even if you have already logged in (by default, this function will exit quickly if you have already logged in)

loginToState

loginToState function

LoginOptions

string

The API key to use. If the parameter is not specified, will try to use the BRAINTRUST_API_KEY environment variable.

string

The URL of the Braintrust App. Defaults to https://www.braintrust.dev. You should not need to change this unless you are doing the “Full” deployment.

boolean

If true, disables the local span cache used to optimize scorer access to trace data. When disabled, scorers will always fetch spans from the server. Defaults to false.

Object

A custom fetch implementation to use.

boolean

By default, the SDK installs an event handler that flushes pending writes on the beforeExit event. If true, this event handler will not be installed.

Object

Calls this function if there’s an error in the background flusher.

string

The name of a specific organization to connect to. Since API keys are scoped to organizations, this parameter is usually unnecessary unless you are logging in with a JWT.

newId

newId function

parseCachedHeader

parseCachedHeader function

undefined | null | string

parseTemplateFormat

parseTemplateFormat function

unknown

TemplateFormat

permalink

Format a permalink to the Braintrust application for viewing the span represented by the provided slug. Links can be generated at any time, but they will only become viewable after the span and its root have been flushed to the server and ingested. If you have a Span object, use Span.link instead.

string

The identifier generated from Span.export.

Object

Optional arguments.

string

The app URL to use. If not provided, the app URL will be inferred from the state.

string

The org name to use. If not provided, the org name will be inferred from the state.

BraintrustState

The login state to use. If not provided, the global state will be used.

promptDefinitionToPromptData

promptDefinitionToPromptData function

unknown

string

required

objectOutputType | objectOutputType | objectOutputType | objectOutputType | objectOutputType

"none" | "mustache" | "nunjucks"

Object[]

registerOtelFlush

Register a callback to flush OTEL spans. This is called by @braintrust/otel when it initializes a BraintrustSpanProcessor/Exporter. When ensureSpansFlushed is called (e.g., before a BTQL query in scorers), this callback will be invoked to ensure OTEL spans are flushed to the server. Also disables the span cache, since OTEL spans aren’t in the local cache and we need BTQL to see the complete span tree (both native + OTEL spans).

Object

renderMessage

renderMessage function

Object

renderPromptParams

renderPromptParams function

Record

Object

boolean

TemplateFormat

renderTemplateContent

renderTemplateContent function

string

Record

Object

boolean

TemplateFormat

Reporter

Reporter function

string

ReporterBody

reportFailures

reportFailures function

EvaluatorDef

string

required

string

required

string

An optional experiment id to use as a base. If specified, the new experiment will be summarized and compared to this experiment. This takes precedence over baseExperimentName if specified.

string

An optional experiment name to use as a base. If specified, the new experiment will be summarized and compared to this experiment.

EvalData

required

A function that returns a list of inputs, expected outputs, and metadata.

string

An optional description for the experiment.

ErrorScoreHandler

string

An optional name for the experiment.

Object

Optional settings for collecting git metadata. By default, will collect all git metadata fields allowed in org-level settings.

boolean

Whether the experiment should be public. Defaults to false.

number

The maximum number of tasks/scorers that will be run concurrently. Defaults to undefined, in which case there is no max concurrency.

Record

Optional additional metadata for the experiment.

Parameters

A set of parameters that will be passed to the evaluator. Can contain array values that will be converted to single values in the task.

string

If specified, uses the given project ID instead of the evaluator’s name to identify the project.

null | Object

Optionally explicitly specify the git metadata for this experiment. This takes precedence over gitMetadataSettings if specified.

EvalScorer[]

required

A set of functions that take an input, output, and expected value and return a score.

AbortSignal

An abort signal that can be used to stop the evaluation.

BraintrustState

If specified, uses the logger state to initialize Braintrust objects. If unspecified, falls back to the global state (initialized using your API key).

boolean

Whether to summarize the scores of the experiment after it has run. Defaults to true.

EvalTask

required

A function that takes an input and returns an output.

number

The duration, in milliseconds, after which to time out the evaluation. Defaults to undefined, in which case there is no timeout.

number

boolean

Whether to update an existing experiment with experiment_name if one exists. Defaults to false.

EvalResult[]

ReporterOpts

runEvaluator

runEvaluator function

null | Experiment

EvaluatorDef

string

required

string

required

string

An optional experiment id to use as a base. If specified, the new experiment will be summarized and compared to this experiment. This takes precedence over baseExperimentName if specified.

string

An optional experiment name to use as a base. If specified, the new experiment will be summarized and compared to this experiment.

EvalData

required

A function that returns a list of inputs, expected outputs, and metadata.

string

An optional description for the experiment.

ErrorScoreHandler

string

An optional name for the experiment.

Object

Optional settings for collecting git metadata. By default, will collect all git metadata fields allowed in org-level settings.

boolean

Whether the experiment should be public. Defaults to false.

number

The maximum number of tasks/scorers that will be run concurrently. Defaults to undefined, in which case there is no max concurrency.

Record

Optional additional metadata for the experiment.

Parameters

A set of parameters that will be passed to the evaluator. Can contain array values that will be converted to single values in the task.

string

If specified, uses the given project ID instead of the evaluator’s name to identify the project.

null | Object

Optionally explicitly specify the git metadata for this experiment. This takes precedence over gitMetadataSettings if specified.

EvalScorer[]

required

A set of functions that take an input, output, and expected value and return a score.

AbortSignal

An abort signal that can be used to stop the evaluation.

BraintrustState

If specified, uses the logger state to initialize Braintrust objects. If unspecified, falls back to the global state (initialized using your API key).

boolean

Whether to summarize the scores of the experiment after it has run. Defaults to true.

EvalTask

required

A function that takes an input and returns an output.

number

The duration, in milliseconds, after which to time out the evaluation. Defaults to undefined, in which case there is no timeout.

number

boolean

Whether to update an existing experiment with experiment_name if one exists. Defaults to false.

ProgressReporter

Filter[]

undefined | Object

InferParameters

boolean

setFetch

Set the fetch implementation to use for requests. You can specify it here, or when you call login.

Object

The fetch implementation to use.

setMaskingFunction

Set a global masking function that will be applied to all logged data before sending to Braintrust. The masking function will be applied after records are merged but before they are sent to the backend.

null | Object

A function that takes a JSON-serializable object and returns a masked version. Set to null to disable masking.

spanComponentsToObjectId

spanComponentsToObjectId function

Object

SpanComponentsV3

required

BraintrustState

startSpan

Lower-level alternative to traced. This allows you to start a span yourself, and can be useful in situations where you cannot use callbacks. However, spans started with startSpan will not be marked as the “current span”, so currentSpan() and traced() will be no-ops. If you want to mark a span as current, use traced instead. See traced for full details.

unknown

StartSpanEventArgs

string

ParentSpanIds | MultiParentSpanIds

StartSpanEventArgs

Record

string

number

SpanType

summarize

Summarize the current experiment, including the scores (compared to the closest reference experiment) and metadata.

Object

Options for summarizing the experiment.

string

The experiment to compare against. If None, the most recent experiment on the origin’s main branch will be used.

boolean

Whether to summarize the scores. If False, only the metadata will be returned.

traceable

A synonym for wrapTraced. If you’re porting from systems that use traceable, you can use this to make your codebase more consistent.

unknown

StartSpanEventArgs

string

ParentSpanIds | MultiParentSpanIds

StartSpanEventArgs

Record

string

number

SpanType

boolean

traced

Toplevel function for starting a span. It checks the following (in precedence order):

Currently-active span
Currently-active experiment
Currently-active logger

and creates a span under the first one that is active. Alternatively, if parent is specified, it creates a span under the specified parent row. If none of these are active, it returns a no-op span object. See Span.traced for full details.

Object

unknown

StartSpanEventArgs

string

ParentSpanIds | MultiParentSpanIds

StartSpanEventArgs

Record

string

number

SpanType

boolean

updateSpan

Update a span using the output of span.export(). It is important that you only resume updating to a span once the original span has been fully written and flushed, since otherwise updates to the span may conflict with the original span.

unknown

string

required

withCurrent

Runs the provided callback with the span as the current span.

Span

string

required

Row ID of the span.

"span"

required

string

required

Root span ID of the span.

string

required

Span ID of the span.

string[]

required

Parent span IDs of the span.

Object

undefined | BraintrustState

withDataset

withDataset function

string

Object

Readonly

string

The API key to use. If the parameter is not specified, will try to use the BRAINTRUST_API_KEY environment variable.

string

The URL of the Braintrust App. Defaults to https://www.braintrust.dev. You should not need to change this unless you are doing the “Full” deployment.

boolean

If true, disables the local span cache used to optimize scorer access to trace data. When disabled, scorers will always fetch spans from the server. Defaults to false.

Object

A custom fetch implementation to use.

boolean

By default, the SDK installs an event handler that flushes pending writes on the beforeExit event. If true, this event handler will not be installed.

Object

Calls this function if there’s an error in the background flusher.

string

The name of a specific organization to connect to. Since API keys are scoped to organizations, this parameter is usually unnecessary unless you are logging in with a JWT.

string

Record

string

BraintrustState

string

withExperiment

withExperiment function

string

Object

Readonly

string

The API key to use. If the parameter is not specified, will try to use the BRAINTRUST_API_KEY environment variable.

string

The URL of the Braintrust App. Defaults to https://www.braintrust.dev. You should not need to change this unless you are doing the “Full” deployment.

boolean

If true, disables the local span cache used to optimize scorer access to trace data. When disabled, scorers will always fetch spans from the server. Defaults to false.

Object

A custom fetch implementation to use.

boolean

By default, the SDK installs an event handler that flushes pending writes on the beforeExit event. If true, this event handler will not be installed.

Object

Calls this function if there’s an error in the background flusher.

string

The name of a specific organization to connect to. Since API keys are scoped to organizations, this parameter is usually unnecessary unless you are logging in with a JWT.

string

DatasetRef | AnyDataset

string

Object

boolean

Record

string

null | Object

boolean

BraintrustState

boolean

withLogger

withLogger function

Object

Readonly

string

The API key to use. If the parameter is not specified, will try to use the BRAINTRUST_API_KEY environment variable.

string

The URL of the Braintrust App. Defaults to https://www.braintrust.dev. You should not need to change this unless you are doing the “Full” deployment.

boolean

If true, disables the local span cache used to optimize scorer access to trace data. When disabled, scorers will always fetch spans from the server. Defaults to false.

Object

A custom fetch implementation to use.

boolean

By default, the SDK installs an event handler that flushes pending writes on the beforeExit event. If true, this event handler will not be installed.

Object

Calls this function if there’s an error in the background flusher.

string

The name of a specific organization to connect to. Since API keys are scoped to organizations, this parameter is usually unnecessary unless you are logging in with a JWT.

OrgProjectMetadata

string

boolean

BraintrustState

boolean

withParent

withParent function

string

Object

undefined | BraintrustState

wrapAgentClass

wrapAgentClass function

any

WrapAISDKOptions

wrapAISDK

Wraps Vercel AI SDK methods with Braintrust tracing. Returns wrapped versions of generateText, streamText, generateObject, streamObject, Agent, experimental_Agent, and ToolLoopAgent that automatically create spans and log inputs, outputs, and metrics.

WrapAISDKOptions

wrapAISDKModel

Wrap an ai-sdk model (created with .chat(), .completion(), etc.) to add tracing. If Braintrust is not configured, this is a no-op

wrapAnthropic

Wrap an Anthropic object (created with new Anthropic(...)) to add tracing. If Braintrust is not configured, nothing will be traced. If this is not an Anthropic object, this function is a no-op. Currently, this only supports the v4 API.

wrapClaudeAgentSDK

Wraps the Claude Agent SDK with Braintrust tracing. This returns wrapped versions of query and tool that automatically trace all agent interactions.

The Claude Agent SDK module

wrapGoogleGenAI

Wrap a Google GenAI module (imported with import * as googleGenAI from '@google/genai') to add tracing. If Braintrust is not configured, nothing will be traced.

The Google GenAI module

wrapMastraAgent

wrapMastraAgent function

Object

string

wrapOpenAI

Wrap an OpenAI object (created with new OpenAI(...)) to add tracing. If Braintrust is not configured, nothing will be traced. If this is not an OpenAI object, this function is a no-op. Currently, this supports both the v4 and v5 API.

wrapOpenAIv4

wrapOpenAIv4 function

wrapTraced

Wrap a function with traced, using the arguments as input and return value as output. Any functions wrapped this way will automatically be traced, similar to the @traced decorator in Python. If you want to correctly propagate the function’s name and define it in one go, then you can do so like this:

const myFunc = wrapTraced(async function myFunc(input) {
 const result = await client.chat.completions.create({
   model: "gpt-3.5-turbo",
   messages: [{ role: "user", content: input }],
 });
 return result.choices[0].message.content ?? "unknown";
},
// Optional: if you're using a framework like NextJS that minifies your code, specify the function name and it will be used for the span name
{ name: "myFunc" },
);

Now, any calls to myFunc will be traced, and the input and output will be logged automatically. If tracing is inactive, i.e. there is no active logger or experiment, it’s just a no-op.

The function to wrap.

unknown

Span-level arguments (e.g. a custom name or type) to pass to traced.

StartSpanEventArgs

string

ParentSpanIds | MultiParentSpanIds

StartSpanEventArgs

Record

string

number

SpanType

boolean

Classes

Attachment

Represents an attachment to be uploaded and the associated metadata. Attachment objects can be inserted anywhere in an event, allowing you to log arbitrary file data. The SDK will asynchronously upload the file to object storage and replace the Attachment object with an AttachmentReference. Properties

Object

The object that replaces this Attachment at upload time.

Methods data(), debugInfo(), upload()

BaseAttachment

BaseAttachment class Properties

Object | Object

Methods data(), debugInfo(), upload()

BraintrustState

BraintrustState class Properties

null | string

undefined | Experiment

undefined | Logger

IsoAsyncLocalStorage

Object

string

boolean

null | string

PromptCache

null | string

SpanCache

unknown

Methods apiConn(), appConn(), bgLogger(), copyLoginInfo(), disable(), enforceQueueSizeLimit(), flushOtel(), httpLogger(), login(), loginReplaceApiConn(), proxyConn(), registerOtelFlush(), resetIdGenState(), resetLoginInfo(), serialize(), setFetch(), setMaskingFunction(), setOverrideBgLogger(), toJSON(), toString(), deserialize()

BraintrustStream

A Braintrust stream. This is a wrapper around a ReadableStream of BraintrustStreamChunk, with some utility methods to make them easy to log and convert into various formats. Methods [asyncIterator](), copy(), finalValue(), toReadableStream(), parseRawEvent(), serializeRawEvent()

CachedSpanFetcher

Cached span fetcher that handles fetching and caching spans by type. Caching strategy:

Cache spans by span type (Map<spanType, SpanData[]>)
Track if all spans have been fetched (allFetched flag)
When filtering by spanType, only fetch types not already in cache

Methods getSpans()

CodeFunction

CodeFunction class Properties

string

"replace" | "error" | "ignore"

Record

string

ZodType

Project

ZodType

string

Methods key()

CodePrompt

CodePrompt class Properties

string

string

"replace" | "error" | "ignore"

Record

string

Project

Object

string

Object | Object | GenericCodeFunction[]

Methods toFunctionDefinition()

ContextManager

ContextManager class Methods getCurrentSpan(), getParentSpanIds(), runInContext()

Dataset

A dataset is a collection of records, such as model inputs and expected outputs, which represent data you can use to evaluate and fine-tune models. You can log production data to datasets, curate them with interesting examples, edit/delete records, and run evaluations against them. You should not create Dataset objects directly. Instead, use the braintrust.initDataset() method. Properties

unknown

Methods [asyncIterator](), clearCache(), close(), delete(), fetch(), fetchedData(), flush(), getState(), insert(), summarize(), update(), version(), isDataset()

EvalResultWithSummary

EvalResultWithSummary class Properties

EvalResult[]

ExperimentSummary

Methods toJSON(), toString()

Experiment

An experiment is a collection of logged events, such as model inputs and outputs, which represent a snapshot of your application at a particular point in time. An experiment is meant to capture more than just the model you use, and includes the data you use to test, pre- and post- processing code, comparison metrics (scores), and any other metadata you want to include. Experiments are associated with a project, and two experiments are meant to be easily comparable via their inputs. You can change the attributes of the experiments in a project (e.g. scoring functions) over time, simply by changing what you log. You should not create Experiment objects directly. Instead, use the braintrust.init() method. Properties

AnyDataset

"experiment"

unknown

Methods [asyncIterator](), clearCache(), close(), export(), fetch(), fetchBaseExperiment(), fetchedData(), flush(), getState(), log(), logFeedback(), startSpan(), summarize(), traced(), updateSpan(), version()

ExternalAttachment

Represents an attachment that resides in an external object store and the associated metadata. ExternalAttachment objects can be inserted anywhere in an event, similar to Attachment objects, but they reference files that already exist in an external object store rather than requiring upload. The SDK will replace the ExternalAttachment object with an AttachmentReference during logging. Properties

Object

The object that replaces this ExternalAttachment at upload time.

Methods data(), debugInfo(), upload()

FailedHTTPResponse

FailedHTTPResponse class Properties

string

number

string

IDGenerator

Abstract base class for ID generators Methods getSpanId(), getTraceId(), shareRootSpanId()

JSONAttachment

Represents a JSON object that should be stored as an attachment. JSONAttachment is a convenience function that creates an Attachment from JSON data. It’s particularly useful for large JSON objects that would otherwise bloat the trace size. The JSON data is automatically serialized and stored as an attachment with content type “application/json”. Properties

Object

The object that replaces this Attachment at upload time.

Methods data(), debugInfo(), upload()

LazyValue

LazyValue class Properties

unknown

Methods get(), getSync()

Logger

Logger class Properties

"logger"

unknown

Methods export(), flush(), log(), logFeedback(), startSpan(), traced(), updateSpan()

LoginInvalidOrgError

LoginInvalidOrgError class Properties

string

NoopSpan

A fake implementation of the Span API which does nothing. This can be used as the default span. Properties

string

Row ID of the span.

"span"

string

Root span ID of the span.

string

Span ID of the span.

string[]

Parent span IDs of the span.

Methods close(), end(), export(), flush(), getParentInfo(), link(), log(), logFeedback(), permalink(), setAttributes(), startSpan(), startSpanWithParents(), state(), toString(), traced()

ObjectFetcher

ObjectFetcher class Properties

unknown

Methods [asyncIterator](), clearCache(), fetch(), fetchedData(), getState(), version()

Project

Project class Properties

string

PromptBuilder

ScorerBuilder

ToolBuilder

Methods addCodeFunction(), addPrompt(), publish()

ProjectNameIdMap

ProjectNameIdMap class Methods getId(), getName(), resolve()

Prompt

Prompt class Properties

unknown

Methods build(), buildWithAttachments(), fromPromptData(), isPrompt(), renderPrompt()

PromptBuilder

PromptBuilder class Methods create()

ReadonlyAttachment

A readonly alternative to Attachment, which can be used for fetching already-uploaded Attachments. Properties

Object | Object

Attachment metadata.

Methods asBase64Url(), data(), metadata(), status()

ReadonlyExperiment

A read-only view of an experiment, initialized by passing open: true to init(). Properties

unknown

Methods [asyncIterator](), asDataset(), clearCache(), fetch(), fetchedData(), getState(), version()

ScorerBuilder

ScorerBuilder class Methods create()

SpanFetcher

Fetcher for spans by root_span_id, using the ObjectFetcher pattern. Handles pagination automatically via cursor-based iteration. Properties

unknown

Methods [asyncIterator](), clearCache(), fetch(), fetchedData(), getState(), version()

SpanImpl

Primary implementation of the Span interface. See Span for full details on each method. We suggest using one of the various traced methods, instead of creating Spans directly. See Span.startSpan for full details. Properties

"span"

unknown

Row ID of the span.

unknown

Root span ID of the span.

unknown

Span ID of the span.

unknown

Parent span IDs of the span.

Methods close(), end(), export(), flush(), getParentInfo(), link(), log(), logFeedback(), permalink(), setAttributes(), setSpanParents(), startSpan(), startSpanWithParents(), state(), toString(), traced()

TestBackgroundLogger

TestBackgroundLogger class Methods drain(), flush(), log(), setMaskingFunction()

ToolBuilder

ToolBuilder class Methods create()

UUIDGenerator

ID generator that uses UUID4 for both span and trace IDs Methods getSpanId(), getTraceId(), shareRootSpanId()

Interfaces

AttachmentParams

AttachmentParams interface Properties

string

string | ArrayBuffer | Blob

string

BraintrustState

BackgroundLoggerOpts

BackgroundLoggerOpts interface Properties

boolean

Object

ContextParentSpanIds

ContextParentSpanIds interface Properties

string

string[]

DatasetSummary

Summary of a dataset’s scores and metadata. Properties

undefined | DataSummary

Summary of the dataset’s data.

string

Name of the dataset.

string

URL to the experiment’s page in the Braintrust app.

string

Name of the project that the dataset belongs to.

string

URL to the project’s page in the Braintrust app.

DataSummary

Summary of a dataset’s data. Properties

number

New or updated records added in this session.

number

Total records in the dataset.

EvalHooks

EvalHooks interface Properties

Expected

The expected output for the current evaluation.

Object

unknown

The metadata object for the current evaluation. You can mutate this object to add or remove metadata.

InferParameters

The current parameters being used for this specific task execution. Array parameters are converted to single values.

Object

Report progress that will show up in the playground.

Span

The task’s span.

undefined | string[]

The tags for the current evaluation.

number

The index of the current trial (0-based). This is useful when trialCount > 1.

Evaluator

Evaluator interface Properties

string

An optional experiment id to use as a base. If specified, the new experiment will be summarized and compared to this experiment. This takes precedence over baseExperimentName if specified.

string

An optional experiment name to use as a base. If specified, the new experiment will be summarized and compared to this experiment.

EvalData

A function that returns a list of inputs, expected outputs, and metadata.

string

An optional description for the experiment.

ErrorScoreHandler

string

An optional name for the experiment.

Object

Optional settings for collecting git metadata. By default, will collect all git metadata fields allowed in org-level settings.

boolean

Whether the experiment should be public. Defaults to false.

number

The maximum number of tasks/scorers that will be run concurrently. Defaults to undefined, in which case there is no max concurrency.

Record

Optional additional metadata for the experiment.

Parameters

A set of parameters that will be passed to the evaluator. Can contain array values that will be converted to single values in the task.

string

If specified, uses the given project ID instead of the evaluator’s name to identify the project.

null | Object

Optionally explicitly specify the git metadata for this experiment. This takes precedence over gitMetadataSettings if specified.

EvalScorer[]

A set of functions that take an input, output, and expected value and return a score.

AbortSignal

An abort signal that can be used to stop the evaluation.

BraintrustState

If specified, uses the logger state to initialize Braintrust objects. If unspecified, falls back to the global state (initialized using your API key).

boolean

Whether to summarize the scores of the experiment after it has run. Defaults to true.

EvalTask

A function that takes an input and returns an output.

number

The duration, in milliseconds, after which to time out the evaluation. Defaults to undefined, in which case there is no timeout.

number

boolean

Whether to update an existing experiment with experiment_name if one exists. Defaults to false.

ExperimentSummary

Summary of an experiment’s scores and metadata. Properties

string

The experiment scores are baselined against.

string

ID of the experiment. May be undefined if the eval was run locally.

string

Name of the experiment.

string

URL to the experiment’s page in the Braintrust app.

Record

string

Name of the project that the experiment belongs to.

string

URL to the project’s page in the Braintrust app.

Record

Summary of the experiment’s scores.

Exportable

Exportable interface

ExternalAttachmentParams

ExternalAttachmentParams interface Properties

string

BraintrustState

string

FunctionEvent

FunctionEvent interface Properties

string

"replace" | "error" | "ignore"

Record

string

Object

string

InvokeFunctionArgs

Arguments for the invoke function. Properties

The type of the global function to invoke. If unspecified, defaults to ‘scorer’ for backward compatibility.

string

The ID of the function to invoke.

string

The name of the global function to invoke.

Input

The input to the function. This will be logged as the input field in the span.

Additional OpenAI-style messages to add to the prompt (only works for llm functions).

Record

null | "text" | "auto" | "parallel" | "json"

string | Exportable

string

The ID of the project to use for execution context (API keys, project defaults, etc.). This is not the project the function belongs to, but the project context for the invocation.

string

The name of the project containing the function to invoke.

string

The ID of the function in the prompt session to invoke.

string

The ID of the prompt session to invoke the function from.

unknown

A Zod schema to validate the output of the function and return a typed value. This is only used if stream is false.

string

The slug of the function to invoke.

BraintrustState

(Advanced) This parameter allows you to pass in a custom login state. This is useful for multi-tenant environments where you are running functions from different Braintrust organizations.

Stream

Whether to stream the function’s output. If true, the function will return a BraintrustStream, otherwise it will return the output of the function as a JSON object.

boolean

Whether to use strict mode for the function. If true, the function will throw an error if the variable names in the prompt do not match the input keys.

string[]

Tags to add to the span. This will be logged as the tags field in the span.

string

The version of the function to invoke.

LoginOptions

Options for logging in to Braintrust. Properties

string

The API key to use. If the parameter is not specified, will try to use the BRAINTRUST_API_KEY environment variable.

string

The URL of the Braintrust App. Defaults to https://www.braintrust.dev. You should not need to change this unless you are doing the “Full” deployment.

boolean

If true, disables the local span cache used to optimize scorer access to trace data. When disabled, scorers will always fetch spans from the server. Defaults to false.

Object

A custom fetch implementation to use.

boolean

By default, the SDK installs an event handler that flushes pending writes on the beforeExit event. If true, this event handler will not be installed.

Object

Calls this function if there’s an error in the background flusher.

string

The name of a specific organization to connect to. Since API keys are scoped to organizations, this parameter is usually unnecessary unless you are logging in with a JWT.

LogOptions

LogOptions interface Properties

IsAsyncFlush

Record

LinkArgs

MetricSummary

Summary of a metric’s performance. Properties

number

Difference in metric between the current and reference experiment.

number

Number of improvements in the metric.

number

Average metric across all examples.

string

Name of the metric.

number

Number of regressions in the metric.

string

Unit label for the metric.

ObjectMetadata

ObjectMetadata interface Properties

Record

string

ParentExperimentIds

ParentExperimentIds interface Properties

string

ParentProjectLogIds

ParentProjectLogIds interface Properties

"g"

string

ReporterBody

ReporterBody interface

ScoreSummary

Summary of a score’s performance. Properties

number

Difference in score between the current and reference experiment.

number

Number of improvements in the score.

string

Name of the score.

number

Number of regressions in the score.

number

Average score across all examples.

Span

A Span encapsulates logged data and metrics for a unit of work. This interface is shared by all span implementations. We suggest using one of the various traced methods, instead of creating Spans directly. See Span.traced for full details. Properties

string

Row ID of the span.

"span"

string

Root span ID of the span.

string

Span ID of the span.

string[]

Parent span IDs of the span.

SpanData

Span data returned by getSpans(). Properties

unknown

Record

unknown

Object

string

string[]

Trace

Interface for trace objects that can be used by scorers. Both the SDK’s LocalTrace class and the API wrapper’s WrapperTrace implement this.

Braintrust TypeScript SDK

⌘I

​Installation

​Functions

​BaseExperiment

​BraintrustMiddleware

​buildLocalSummary

​createFinalValuePassThroughStream

​currentExperiment

​currentLogger

​currentSpan

​defaultErrorScoreHandler

​deserializePlainStringAsJSON

​devNullWritableStream

​Eval

​flush

​getContextManager

​getIdGenerator

​getPromptVersions

​getSpanParentObject

​init

​initDataset

​initExperiment

​initFunction

​initLogger

​invoke

​isTemplateFormat

​loadPrompt

​log

​logError

​login

​loginToState

​newId

​parseCachedHeader

​parseTemplateFormat

​permalink

​promptDefinitionToPromptData

​registerOtelFlush

​renderMessage

​renderPromptParams

​renderTemplateContent

​Reporter

​reportFailures

​runEvaluator

​setFetch

​setMaskingFunction

​spanComponentsToObjectId

​startSpan

​summarize

​traceable

​traced

​updateSpan

​withCurrent

​withDataset

​withExperiment

​withLogger

​withParent

​wrapAgentClass

​wrapAISDK

​wrapAISDKModel

​wrapAnthropic

​wrapClaudeAgentSDK

​wrapGoogleGenAI

​wrapMastraAgent

​wrapOpenAI

​wrapOpenAIv4

​wrapTraced

​Classes

​Attachment

​BaseAttachment

​BraintrustState

​BraintrustStream

​CachedSpanFetcher

​CodeFunction

​CodePrompt

​ContextManager

​Dataset

​EvalResultWithSummary

​Experiment

​ExternalAttachment

​FailedHTTPResponse

​IDGenerator

Installation

Functions

BaseExperiment

BraintrustMiddleware

buildLocalSummary

createFinalValuePassThroughStream

currentExperiment

currentLogger

currentSpan

defaultErrorScoreHandler

deserializePlainStringAsJSON

devNullWritableStream

Eval

flush

getContextManager

getIdGenerator

getPromptVersions

getSpanParentObject

init

initDataset

initExperiment

initFunction

initLogger

invoke

isTemplateFormat

loadPrompt

log

logError

login

loginToState

newId

parseCachedHeader

parseTemplateFormat

permalink

promptDefinitionToPromptData

registerOtelFlush

renderMessage

renderPromptParams

renderTemplateContent

Reporter

reportFailures

runEvaluator

setFetch

setMaskingFunction

spanComponentsToObjectId

startSpan

summarize

traceable

traced

updateSpan

withCurrent

withDataset

withExperiment

withLogger

withParent

wrapAgentClass

wrapAISDK

wrapAISDKModel

wrapAnthropic

wrapClaudeAgentSDK

wrapGoogleGenAI

wrapMastraAgent

wrapOpenAI

wrapOpenAIv4

wrapTraced

Classes

Attachment

BaseAttachment

BraintrustState

BraintrustStream

CachedSpanFetcher

CodeFunction

CodePrompt

ContextManager

Dataset

EvalResultWithSummary

Experiment

ExternalAttachment

FailedHTTPResponse

IDGenerator