Braintrust Python SDK

This page provides a complete, auto-generated reference for the Braintrust Python SDK. For usage guidance on important APIs, see the API Reference. Also see the Braintrust Python SDK on GitHub.

Requirements

Python 3.10 or higher

Installation

pip install braintrust

Functions

Eval

A function you can use to define an evaluator. This is a convenience wrapper around the Evaluator class.

str

The name of the evaluator. This corresponds to a project name in Braintrust.

EvalData[Input, Expected]

Returns an iterator over the evaluation dataset. Each element of the iterator should be a EvalCase.

EvalTask[Input, Output, Expected]

Runs the evaluation task on a single input. The hooks object can be used to add metadata to the evaluation.

Sequence[EvalScorer[Input, Output, Expected]] | None

A list of scorers to evaluate the results of the task. Each scorer can be a Scorer object or a function that takes an EvalScorerArgs object and returns a Score object.

Sequence[EvalClassifier[Input, Output, Expected]] | None

str | None

(Optional) Experiment name. If not specified, a name will be generated automatically.

int

The number of times to run the evaluator per input. This is useful for evaluating applications that have non-deterministic behavior and gives you both a stronger aggregate measure and a sense of the variance in the results.

Metadata | None

(Optional) A dictionary with additional data about the test example, model outputs, or just about anything else that’s relevant, that you can use to help find and analyze examples later. For example, you could log the prompt, example’s id, or anything else that would be useful to slice/dice later. The values in metadata can be any JSON-serializable type, but its keys must be strings.

Sequence[str] | None

(Optional) A list of tags to associate with the experiment.

bool

(Optional) Whether the experiment should be public. Defaults to false.

bool

ReporterDef[Input, Output, Expected, EvalReport] | None

(Optional) A reporter that takes an evaluator and its result and returns a report.

str

required

Callable[[Evaluator[Input, Output, Expected], EvalResultWithSummary[Input, Output, Expected], bool, bool], EvalReport | Awaitable[EvalReport]]

required

Callable[[list[EvalReport], bool, bool], bool | Awaitable[bool]]

required

float | None

(Optional) The duration, in seconds, after which to time out the evaluation. Defaults to None, in which case there is no timeout.

int | None

str | None

(Optional) If specified, uses the given project ID instead of the evaluator’s name to identify the project.

str | None

An optional experiment name to use as a base. If specified, the new experiment will be summarized and compared to this experiment.

str | None

An optional experiment id to use as a base. If specified, the new experiment will be summarized and compared to this experiment. This takes precedence over base_experiment_name if specified.

GitMetadataSettings | None

Optional settings for collecting git metadata. By default, defers to org-level settings returned by the control plane.

Literal['all', 'some', 'none']

required

list[str] | None

required

RepoInfo | None

Optionally explicitly specify the git metadata for this experiment. This takes precedence over git_metadata_settings if specified.

str | None

required

str | None

required

str | None

required

bool | None

required

str | None

required

str | None

required

str | None

required

str | None

required

str | None

required

ErrorScoreHandler[Input, Expected] | None

Optionally supply a custom function to specifically handle score values when tasks or scoring functions have errored.

str | None

An optional description for the experiment.

bool

Whether to summarize the scores of the experiment after it has run.

bool

Do not send logs to Braintrust. When True, the evaluation runs locally and builds a local summary instead of creating an experiment. Defaults to False.

EvalParameters | RemoteEvalParameters | None

A set of parameters that will be passed to the evaluator.

Callable[[ExperimentSummary], None] | None

An optional callback that will be called when the evaluation starts. It receives the ExperimentSummary object, which can be used to display metadata about the experiment.

Callable[[SSEProgressEvent], None] | None

A function that will be called with progress events, which can be used to display intermediate progress.

str | None

If specified, instead of creating a new experiment object, the Eval() will populate the object or span specified by this parent.

BraintrustState | None

Optional BraintrustState to use for the evaluation. If not specified, the global login state will be used.

bool

Whether to enable the span cache for this evaluation. Defaults to True. The span cache stores span data on disk to minimize memory usage and allow scorers to read spans without server round-trips.

EvalAsync

A function you can use to define an evaluator. This is a convenience wrapper around the Evaluator class.

str

The name of the evaluator. This corresponds to a project name in Braintrust.

EvalData[Input, Expected]

Returns an iterator over the evaluation dataset. Each element of the iterator should be a EvalCase.

EvalTask[Input, Output, Expected]

Runs the evaluation task on a single input. The hooks object can be used to add metadata to the evaluation.

Sequence[EvalScorer[Input, Output, Expected]] | None

A list of scorers to evaluate the results of the task. Each scorer can be a Scorer object or a function that takes an EvalScorerArgs object and returns a Score object.

Sequence[EvalClassifier[Input, Output, Expected]] | None

str | None

(Optional) Experiment name. If not specified, a name will be generated automatically.

int

Metadata | None

Sequence[str] | None

(Optional) A list of tags to associate with the experiment.

bool

(Optional) Whether the experiment should be public. Defaults to false.

bool

ReporterDef[Input, Output, Expected, EvalReport] | None

(Optional) A reporter that takes an evaluator and its result and returns a report.

str

required

Callable[[Evaluator[Input, Output, Expected], EvalResultWithSummary[Input, Output, Expected], bool, bool], EvalReport | Awaitable[EvalReport]]

required

Callable[[list[EvalReport], bool, bool], bool | Awaitable[bool]]

required

float | None

(Optional) The duration, in seconds, after which to time out the evaluation. Defaults to None, in which case there is no timeout.

int | None

str | None

(Optional) If specified, uses the given project ID instead of the evaluator’s name to identify the project.

str | None

An optional experiment name to use as a base. If specified, the new experiment will be summarized and compared to this experiment.

str | None

An optional experiment id to use as a base. If specified, the new experiment will be summarized and compared to this experiment. This takes precedence over base_experiment_name if specified.

GitMetadataSettings | None

Optional settings for collecting git metadata. By default, defers to org-level settings returned by the control plane.

Literal['all', 'some', 'none']

required

list[str] | None

required

RepoInfo | None

Optionally explicitly specify the git metadata for this experiment. This takes precedence over git_metadata_settings if specified.

str | None

required

str | None

required

str | None

required

bool | None

required

str | None

required

str | None

required

str | None

required

str | None

required

str | None

required

ErrorScoreHandler[Input, Expected] | None

Optionally supply a custom function to specifically handle score values when tasks or scoring functions have errored.

str | None

An optional description for the experiment.

bool

Whether to summarize the scores of the experiment after it has run.

bool

Do not send logs to Braintrust. When True, the evaluation runs locally and builds a local summary instead of creating an experiment. Defaults to False.

EvalParameters | RemoteEvalParameters | None

A set of parameters that will be passed to the evaluator.

Callable[[ExperimentSummary], None] | None

An optional callback that will be called when the evaluation starts. It receives the ExperimentSummary object, which can be used to display metadata about the experiment.

Callable[[SSEProgressEvent], None] | None

A function that will be called with progress events, which can be used to display intermediate progress.

str | None

If specified, instead of creating a new experiment object, the Eval() will populate the object or span specified by this parent.

BraintrustState | None

Optional BraintrustState to use for the evaluation. If not specified, the global login state will be used.

bool

Whether to enable the span cache for this evaluation. Defaults to True. The span cache stores span data on disk to minimize memory usage and allow scorers to read spans without server round-trips.

Reporter

A function you can use to define a reporter. This is a convenience wrapper around the ReporterDef class.

str

The name of the reporter.

Callable[[Evaluator[Input, Output, Expected], EvalResultWithSummary[Input, Output, Expected], bool, bool], EvalReport | Awaitable[EvalReport]]

A function that takes an evaluator and its result and returns a report.

Callable[[list[EvalReport], bool, bool], bool | Awaitable[bool]]

A function that takes all evaluator results and returns a boolean indicating whether the run was successful.

auto_instrument

Auto-instrument supported AI/ML libraries for Braintrust tracing.

bool

Enable OpenAI instrumentation (default: True)

bool

Enable Anthropic instrumentation (default: True)

bool

Enable LiteLLM instrumentation (default: True)

bool

Enable Pydantic AI instrumentation (default: True)

bool

Enable Google GenAI instrumentation (default: True)

bool

Enable Instructor (structured-output) instrumentation (default: True)

bool

Enable OpenRouter instrumentation (default: True)

bool

Enable Mistral instrumentation (default: True)

bool

Enable HuggingFace Hub instrumentation (default: True)

bool

Enable Agno instrumentation (default: True)

bool

Enable AgentScope instrumentation (default: True)

bool

Enable Claude Agent SDK instrumentation (default: True)

bool

Enable DSPy instrumentation (default: True)

bool

Enable Google ADK instrumentation (default: True)

bool

Enable LangChain instrumentation (default: True)

bool

Enable LlamaIndex instrumentation (default: True)

bool

Enable OpenAI Agents SDK instrumentation (default: True)

bool

Enable Cohere instrumentation (default: True)

bool

Enable AutoGen instrumentation (default: True)

bool

Enable boto3 Bedrock Runtime instrumentation (default: True)

bool

Enable CrewAI instrumentation (default: True)

bool

Enable Strands Agents instrumentation (default: True)

bool

Enable Temporal instrumentation (default: True)

bool

Enable LiveKit Agents instrumentation (default: True)

current_experiment

Returns the currently-active experiment (set by braintrust.init(...)). Returns None if no current experiment has been set.

current_logger

Returns the currently-active logger (set by braintrust.init_logger(...)). Returns None if no current logger has been set.

current_span

Return the currently-active span for logging (set by running a span under a context manager). If there is no active span, returns a no-op span object, which supports the same interface as spans but does no logging.

extract_trace_context

Extract an opaque W3C trace-context from inbound request headers.

dict

Inbound request headers (e.g. an HTTP framework’s headers).

flush

Flush any pending rows to the server.

get_prompt_versions

Get the versions for a specific prompt.

str

The ID of the project to query

str

The ID of the prompt to get versions for

get_span_parent_object

Mainly for internal use. Return the parent object for starting a span in a global context. Applies precedence: current span > propagated parent > experiment > logger.

str | dict | None

BraintrustState | None

init

str | None

The name of the project to create the experiment in. Must specify at least one of project or project_id.

str | None

The name of the experiment to create. If not specified, a name will be generated automatically.

str | None

(Optional) An optional description of the experiment.

Dataset | None | DatasetRef

(Optional) A dataset to associate with the experiment. The dataset must be initialized with braintrust.init_dataset before passing it into the experiment.

RemoteEvalParameters | ParametersRef | None

(Optional) Saved parameters to associate with the experiment. Pass either a RemoteEvalParameters object or a dictionary containing an id and optional version.

str | None

required

str | None

required

str

required

str

required

str | None

required

ParametersSchema

required

dict[str, Any]

required

bool

If the experiment already exists, open it in read-only mode. Throws an error if the experiment does not already exist.

str | None

An optional experiment name to use as a base. If specified, the new experiment will be summarized and compared to this experiment. Otherwise, it will pick an experiment by finding the closest ancestor on the default (e.g. main) branch.

bool

An optional parameter to control whether the experiment is publicly visible to anybody with the link or privately visible to only members of the organization. Defaults to private.

str | None

The URL of the Braintrust App. Defaults to https://www.braintrust.dev.

str | None

The API key to use. If the parameter is not specified, will try to use the BRAINTRUST_API_KEY environment variable. If no API key is specified, will prompt the user to login.

str | None

(Optional) The name of a specific organization to connect to. This is useful if you belong to multiple.

Metadata | None

(Optional) a dictionary with additional data about the test example, model outputs, or just about anything else that’s relevant, that you can use to help find and analyze examples later. For example, you could log the prompt, example’s id, or anything else that would be useful to slice/dice later. The values in metadata can be any JSON-serializable type, but its keys must be strings.

Sequence[str] | None

(Optional) a list of strings to tag the experiment with. Tags can be used to filter and organize experiments.

GitMetadataSettings | None

(Optional) Settings for collecting git metadata. By default, will collect git metadata fields allowed in org-level settings, excluding diff content unless the org opts in.

Literal['all', 'some', 'none']

required

list[str] | None

required

bool

If true (the default), set the global current-experiment to the newly-created one.

bool | None

If the experiment already exists, continue logging to it. If it does not exist, creates the experiment with the specified arguments.

str | None

The id of the project to create the experiment in. This takes precedence over project if specified.

str | None

An optional experiment id to use as a base. If specified, the new experiment will be summarized and compared to this. This takes precedence over base_experiment if specified.

RepoInfo | None

(Optional) Explicitly specify the git metadata for this experiment. This takes precedence over git_metadata_settings if specified.

str | None

required

str | None

required

str | None

required

bool | None

required

str | None

required

str | None

required

str | None

required

str | None

required

str | None

required

BraintrustState | None

(Optional) A BraintrustState object to use. If not specified, will use the global state. This is for advanced use only.

init_dataset

Create a new dataset in a specified project. If the project does not exist, it will be created.

str | None

The name of the dataset to create. If not specified, a name will be generated automatically.

str | None

An optional description of the dataset.

str | int | None

An optional version of the dataset (to read). If not specified, the latest version will be used.

str | None

The URL of the Braintrust App. Defaults to https://www.braintrust.dev.

str | None

The API key to use. If the parameter is not specified, will try to use the BRAINTRUST_API_KEY environment variable. If no API key is specified, will prompt the user to login.

str | None

(Optional) The name of a specific organization to connect to. This is useful if you belong to multiple.

str | None

The id of the project to create the dataset in. This takes precedence over project if specified.

Metadata | None

(Optional) a dictionary, or an object that serializes to a dictionary (such as a Pydantic model), with additional data about the dataset. The values in metadata can be any JSON-serializable type, but its keys must be strings.

bool

(Deprecated) If True, records will be fetched from this dataset in the legacy format, with the “expected” field renamed to “output”. This option will be removed in a future version of Braintrust.

dict[str, Any] | None

(Internal) If specified, the dataset will be created with the given BTQL filters.

BraintrustState | None

(Internal) The Braintrust state to use. If not specified, will use the global state. For advanced use only.

init_experiment

Alias for init

Any

init_function

Creates a function that can be used as either a task or scorer in the Eval framework. When used as a task, it will invoke the specified Braintrust function with the input. When used as a scorer, it will invoke the function with the scorer arguments.

str

The name of the project containing the function.

str

The slug of the function to invoke.

str | None

Optional version of the function to use. Defaults to latest.

init_logger

Create a new logger in a specified project. If the project does not exist, it will be created.

str | None

The name of the project to log into. If unspecified, will default to the Global project.

str | None

The id of the project to log into. This takes precedence over project if specified.

bool

If true (the default), log events will be batched and sent asynchronously in a background thread. If false, log events will be sent synchronously. Set to false in serverless environments.

str | None

The URL of the Braintrust API. Defaults to https://www.braintrust.dev.

str | None

The API key to use. If the parameter is not specified, will try to use the BRAINTRUST_API_KEY environment variable. If no API key is specified, will prompt the user to login.

str | None

(Optional) The name of a specific organization to connect to. This is useful if you belong to multiple.

bool

If true (the default), set the global current-experiment to the newly-created one.

BraintrustState | None

SpanOriginEnvironment | None

inject_trace_context

Inject W3C trace-context headers for the current (or given) span into a carrier.

dict | None

Optional carrier dict (e.g. outbound HTTP headers) to mutate.

Span | None

Optional span to inject. Defaults to the current span.

invoke

Invoke a Braintrust function, returning a BraintrustStream or the value as a plain Python object.

str | None

The ID of the function to invoke.

str | None

The version of the function to invoke.

str | None

The ID of the prompt session to invoke the function from.

str | None

The ID of the function in the prompt session to invoke.

str | None

The name of the project containing the function to invoke.

str | None

The ID of the project to use for execution context (API keys, project defaults, etc.). This is not the project the function belongs to, but the project context for the invocation.

str | None

The slug of the function to invoke.

str | None

The name of the global function to invoke.

FunctionTypeEnum | None

The type of the global function to invoke. If unspecified, defaults to ‘scorer’ for backward compatibility.

Any

The input to the function. This will be logged as the input field in the span.

Sequence[Any] | None

Additional OpenAI-style messages to add to the prompt (only works for llm functions).

Metadata | None

Additional metadata to add to the span. This will be logged as the metadata field in the span. It will also be available as the {{metadata}} field in the prompt and as the metadata argument to the function.

Sequence[str] | None

Tags to add to the span. This will be logged as the tags field in the span.

Exportable | str | None

The parent of the function. This can be an existing span, logger, or experiment, or the output of .export() if you are distributed tracing. If unspecified, will use the same semantics as traced() to determine the parent and no-op if not in a tracing context.

bool

Whether to stream the function’s output. If True, the function will return a BraintrustStream, otherwise it will return the output of the function as a JSON object.

ModeType | None

The response shape of the function if returning tool calls. If “auto”, will return a string if the function returns a string, and a JSON object otherwise. If “parallel”, will return an array of JSON objects with one object per tool call.

bool | None

Whether to use strict mode for the function. If true, the function will throw an error if the variable names in the prompt do not match the input keys.

dict[str, Any] | None

Per-call deep-merge into the resolved function data server-side. Useful for facet, code, global, and remote_eval functions (for example, overriding a facet’s model or a global function’s config). Has no effect on prompt functions, whose parameters live on a separate field that the override path does not touch.

str | None

The name of the Braintrust organization to use.

str | None

The API key to use for authentication.

str | None

The URL of the Braintrust application.

Whether to force a new login even if already logged in.

load_parameters

Load saved parameters from Braintrust.

str | None

The name of the project to load the parameters from. Must specify at least one of project or project_id.

str | None

The slug of the parameters to load.

str | int | None

An optional version of the parameters to read. If not specified, the latest version will be used.

str | None

The ID of the project to load the parameters from. This takes precedence over project.

str | None

The ID of a specific parameters object to load. If specified, this takes precedence over project and slug.

str | None

The environment to load the parameters from. If both version and environment are provided, version takes precedence.

str | None

The URL of the Braintrust App. Defaults to https://www.braintrust.dev.

str | None

The API key to use. If the parameter is not specified, will try to use the BRAINTRUST_API_KEY environment variable.

str | None

The name of a specific organization to connect to.

load_prompt

Loads a prompt from the specified project.

str | None

The name of the project to load the prompt from. Must specify at least one of project or project_id.

str | None

The slug of the prompt to load.

str | int | None

An optional version of the prompt (to read). If not specified, the latest version will be used.

str | None

The id of the project to load the prompt from. This takes precedence over project if specified.

str | None

The id of a specific prompt to load. If specified, this takes precedence over all other parameters (project, slug, version).

Mapping[str, Any] | None

(Optional) A dictionary of default values to use when rendering the prompt. Prompt values will override these defaults.

bool

If true, do not include logging metadata for this prompt when build() is called.

str | None

The environment to load the prompt from. If both version and environment are provided, version takes precedence.

str | None

The URL of the Braintrust App. Defaults to https://www.braintrust.dev.

str | None

The API key to use. If the parameter is not specified, will try to use the BRAINTRUST_API_KEY environment variable. If no API key is specified, will prompt the user to login.

str | None

(Optional) The name of a specific organization to connect to. This is useful if you belong to multiple.

load_prompt_async

Asynchronously loads a prompt from the specified project.

str | None

str | int | None

str | None

Mapping[str, Any] | None

bool

str | None

log

Log a single event to the current experiment. The event will be batched and uploaded behind the scenes.

Any

Log into Braintrust. This will prompt you for your API token, which you can find at https://www.braintrust.dev/app/token. This method is called automatically by init().

str | None

The URL of the Braintrust App. Defaults to https://www.braintrust.dev.

str | None

The API key to use. If the parameter is not specified, will try to use the BRAINTRUST_API_KEY environment variable. If no API key is specified, will prompt the user to login.

str | None

(Optional) The name of a specific organization to connect to. This is useful if you belong to multiple.

Login again, even if you have already logged in (by default, this function will exit quickly if you have already logged in)

parent_context

Context manager to temporarily set the parent context for spans.

str | dict | None

The parent to set during the context. May be an exported slug string or an opaque W3C trace-context dict (from extract_trace_context).

BraintrustState | None

Optional BraintrustState to use. If not provided, uses the global state.

parse_stream

Parse a BraintrustStream into its final value.

BraintrustStream

The BraintrustStream to parse.

patch_openai

Patch OpenAI globally for Braintrust tracing.

permalink

Format a permalink to the Braintrust application for viewing the span represented by the provided slug.

str

The identifier generated from Span.export.

str | None

The org name to use. If not provided, the org name will be inferred from the global login state.

str | None

The app URL to use. If not provided, the app URL will be inferred from the global login state.

register_otel_flush

Register a callback to flush OTEL spans. This is called by the OTEL integration when it initializes a span processor/exporter.

Any

The async callback function to flush OTEL spans.

run_evaluator

Wrapper on _run_evaluator_internal that times out execution after evaluator.timeout.

Experiment | None

Evaluator[Input, Output, Expected]

str

required

str

required

EvalData[Input, Expected]

required

EvalTask[Input, Output, Expected]

required

Sequence[EvalScorer[Input, Output, Expected]]

required

str | None

required

Metadata | None

required

Sequence[str] | None

required

int

required

bool

required

bool

required

float | None

required

int | None

required

str | None

required

str | None

required

str | None

required

GitMetadataSettings | None

required

RepoInfo | None

required

ErrorScoreHandler[Input, Expected] | None

required

str | None

required

bool

required

EvalParameters | RemoteEvalParameters | None

required

list[EvalClassifier[Input, Output, Expected]] | None

required

dict[str, Any] | None

required

int | None

list[Filter]

Callable[[SSEProgressEvent], None] | None

BraintrustState | None

bool

set_http_adapter

Specify a custom HTTP adapter to use for all network requests. This is useful for setting custom retry policies, timeouts, etc. Braintrust uses the requests library, so the adapter should be an instance of requests.adapters.HTTPAdapter. Alternatively, consider sub-classing our RetryRequestExceptionsAdapter to get automatic retries on network-related exceptions.

HTTPAdapter

The adapter to use.

set_masking_function

Set a global masking function that will be applied to all logged data before sending to Braintrust. The masking function will be applied after records are merged but before they are sent to the backend.

Callable[[Any], Any] | None

A function that takes a JSON-serializable object and returns a masked version. Set to None to disable masking.

set_thread_pool_max_workers

Set the maximum number of threads to use for running evaluators. By default, this is the number of CPUs on the machine.

Any

span_components_to_object_id

Utility function to resolve the object ID of a SpanComponentsV4 object. This function may trigger a login to braintrust if the object ID is encoded lazily.

SpanComponentsV4

SpanObjectTypeV3

required

str | None

required

dict | None

required

str | None

required

str | None

required

str | None

required

dict | None

required

start_span

Lower-level alternative to @traced for starting a span at the toplevel. It creates a span under the first active object (using the same precedence order as @traced), or if parent is specified, under the specified parent row, or returns a no-op span object.

str | None

SpanTypeAttribute | None

SpanAttributes | Mapping[str, Any] | None

float | None

bool | None

str | dict | None

dict[str, Any] | None

BraintrustState | None

SpanInternalOptions | None

Any

summarize

Summarize the current experiment, including the scores (compared to the closest reference experiment) and metadata.

bool

Whether to summarize the scores. If False, only the metadata will be returned.

str | None

The experiment to compare against. If None, the most recent experiment on the comparison_commit will be used.

traced

Decorator to trace the wrapped function. Can either be applied bare (@traced) or by providing arguments (@traced(*span_args, **span_kwargs)), which will be forwarded to the created span. See Span.start_span for full details on the span arguments.

Any

update_span

Update a span using the output of span.export(). It is important that you only resume updating to a span once the original span has been fully written and flushed, since otherwise updates to the span may conflict with the original span.

str

The output of span.export().

Any

Classes

AsyncScorerLike

Protocol for asynchronous scorers that implement the eval_async interface. The framework will prefer this interface if available. Methods eval_async()

Attachment

Represents an attachment to be uploaded and the associated metadata. Properties

AttachmentReference

The object that replaces this Attachment at upload time.

bytes

The attachment contents. This is a lazy value that will read the attachment contents from disk or memory on first access.

Methods __init__(), upload(), debug_info()

BaseExperiment

Use this to specify that the dataset should actually be the data from a previous (base) experiment. If you do not specify a name, Braintrust will automatically figure out the best base experiment to use based on your git history (or fall back to timestamps). Properties

str | None

The name of the base experiment to use. If unspecified, Braintrust will automatically figure out the best base using your git history (or fall back to timestamps).

Methods __init__()

BraintrustConsoleChunk

A console chunk from a Braintrust stream. Properties

str

Literal['stderr', 'stdout']

Literal['console']

Methods __init__()

BraintrustErrorChunk

An error chunk from a Braintrust stream. Properties

str

Literal['error']

Methods __init__()

BraintrustInvokeError

An error that occurs during a Braintrust stream.

BraintrustJsonChunk

A chunk of JSON data from a Braintrust stream. Properties

str

Literal['json_delta']

Methods __init__()

BraintrustProgressChunk

A progress chunk from a Braintrust stream. Properties

str

Literal['json_delta', 'text_delta', 'reasoning_delta']

Literal['progress']

Methods __init__()

BraintrustStream

A Braintrust stream. This is a wrapper around a generator of BraintrustStreamChunk, with utility methods to make them easy to log and convert into various formats. Properties

Iterable[BraintrustStreamChunk]

Methods __init__(), copy(), final_value()

BraintrustTextChunk

A chunk of text data from a Braintrust stream. Properties

str

Literal['text_delta']

Methods __init__()

ClassifierBuilder

Builder to create a classifier in Braintrust. Properties

Any

Methods __init__(), create()

CodeFunction

A generic callable, with metadata. Properties

Project

Callable[..., Any]

str

str | None

Any

IfExists | None

Metadata | None

Sequence[str] | None

Methods __init__()

CodePrompt

A prompt defined in code, with metadata. Properties

Project

str

PromptData

list[CodeFunction | SavedFunctionId]

str | None

IfExists | None

Metadata | None

Sequence[str] | None

Methods to_function_definition(), __init__()

DataSummary

Summary of a dataset’s data. Properties

int

New or updated records added in this session.

int

Total records in the dataset.

Methods __init__()

Dataset

A dataset is a collection of records, such as model inputs and outputs, which represent data you can use to evaluate and fine-tune models. You can log production data to datasets, curate them with interesting examples, edit/delete records, and run evaluations against them. Properties

Any

str

Any

BraintrustState

Methods __init__(), insert(), update(), delete(), summarize(), close(), flush()

DatasetRef

Reference to a dataset by ID and optional version. Properties

str

DatasetSummary

Summary of a dataset’s scores and metadata. Properties

str

Name of the project that the dataset belongs to.

str

Name of the dataset.

str

URL to the project’s page in the Braintrust app.

str

URL to the experiment’s page in the Braintrust app.

DataSummary | None

Summary of the dataset’s data.

Methods __init__()

EvalCase

An evaluation case. This is a single input to the evaluation task, along with an optional expected output, metadata, and tags. Properties

Input

Expected | None

Metadata | None

Sequence[str] | None

int | None

str | None

Methods __init__()

EvalHooks

An object that can be used to add metadata to an evaluation. This is passed to the task function. Properties

Metadata | None

The metadata object for the current evaluation. You can mutate this object to add or remove metadata.

Expected | None

The expected output for the current evaluation.

Span

Access the span under which the task is run. Also accessible via braintrust.current_span()

int

The index of the current trial (0-based). This is useful when trial_count > 1.

Sequence[str]

The tags for the current evaluation. You can mutate this object to add or remove tags.

ValidatedParameters | None

The parameters for the current evaluation. These are the validated parameter values that were passed to the evaluator.

Methods report_progress(), meta()

EvalResult

The result of an evaluation. This includes the input, expected output, actual output, and metadata. Properties

Input

Output

Mapping[str, float | None]

dict[str, list[ClassificationItem]] | None

Expected | None

Metadata | None

list[str] | None

Exception | None

str | None

Methods __init__()

EvalScorerArgs

Arguments passed to an evaluator scorer. This includes the input, expected output, actual output, and metadata. Properties

Input

Output

Expected | None

Metadata | None

Evaluator

An evaluator is an abstraction that defines an evaluation dataset, a task to run on the dataset, and a set of scorers to evaluate the results of the task. Each method attribute can be synchronous or asynchronous (for optimal performance, it is recommended to provide asynchronous implementations). Properties

str

The name of the project the eval falls under.

str

A name that describes the experiment. You do not need to change it each time the experiment runs.

EvalData[Input, Expected]

Returns an iterator over the evaluation dataset. Each element of the iterator should be an EvalCase or a dict with the same fields as an EvalCase (input, expected, metadata).

EvalTask[Input, Output, Expected]

Runs the evaluation task on a single input. The hooks object can be used to add metadata to the evaluation.

Sequence[EvalScorer[Input, Output, Expected]]

A list of scorers to evaluate the results of the task. Each scorer can be a Scorer object or a function that takes input, output, and expected arguments and returns a Score object. The function can be async.

str | None

Optional experiment name. If not specified, a name will be generated automatically.

Metadata | None

A dictionary with additional data about the test example, model outputs, or just about anything else that’s relevant, that you can use to help find and analyze examples later. For example, you could log the prompt, example’s id, or anything else that would be useful to slice/dice later. The values in metadata can be any JSON-serializable type, but its keys must be strings.

Sequence[str] | None

Optional list of tags for the experiment

int

bool

Whether the experiment should be public. Defaults to false.

bool

Whether to update an existing experiment with experiment_name if one exists. Defaults to false.

float | None

The duration, in seconds, after which to time out the evaluation. Defaults to None, in which case there is no timeout.

int | None

The maximum number of tasks/scorers that will be run concurrently. Defaults to None, in which case there is no max concurrency.

str | None

If specified, uses the given project ID instead of the evaluator’s name to identify the project.

str | None

An optional experiment name to use as a base. If specified, the new experiment will be summarized and compared to this experiment.

str | None

An optional experiment id to use as a base. If specified, the new experiment will be summarized and compared to this experiment. This takes precedence over base_experiment_name if specified.

GitMetadataSettings | None

Optional settings for collecting git metadata. By default, defers entirely to org-level settings returned by the control plane; no git metadata is collected if the org has not configured any.

RepoInfo | None

Optionally explicitly specify the git metadata for this experiment. This takes precedence over git_metadata_settings if specified.

ErrorScoreHandler[Input, Expected] | None

Optionally supply a custom function to specifically handle score values when tasks or scoring functions have errored. A default implementation is exported as default_error_score_handler which will log a 0 score to the root span for any scorer that was not run.

str | None

An optional description for the experiment.

bool

Whether to summarize the scores of the experiment after it has run.

EvalParameters | RemoteEvalParameters | None

A set of parameters that will be passed to the evaluator. Can be used to define prompts or other configurable values.

list[EvalClassifier[Input, Output, Expected]] | None

Optional list of classifiers to evaluate the task output. Classifier results are recorded under the classifications field instead of scores.

dict[str, Any] | None

Methods __init__()

Experiment

An experiment is a collection of logged events, such as model inputs and outputs, which represent a snapshot of your application at a particular point in time. An experiment is meant to capture more than just the model you use, and includes the data you use to test, pre- and post- processing code, comparison metrics (scores), and any other metadata you want to include. Properties

Any

str

Mapping[str, Any]

ObjectMetadata

BraintrustState

Methods __init__(), log(), log_feedback(), start_span(), update_span(), fetch_base_experiment(), summarize(), export(), close(), flush()

ExperimentSummary

Summary of an experiment’s scores and metadata. Properties

str

Name of the project that the experiment belongs to.

str | None

ID of the project. May be None if the eval was run locally.

str | None

ID of the experiment. May be None if the eval was run locally.

str

Name of the experiment.

str | None

URL to the project’s page in the Braintrust app.

str | None

URL to the experiment’s page in the Braintrust app.

str | None

The experiment scores are baselined against.

dict[str, ScoreSummary]

Summary of the experiment’s scores.

dict[str, MetricSummary]

Summary of the experiment’s metrics.

Methods __init__()

ExternalAttachment

Represents an attachment that resides in an external object store and the associated metadata. Properties

AttachmentReference

The object that replaces this Attachment at upload time.

bytes

The attachment contents. This is a lazy value that will read the attachment contents from the external object store on first access.

Methods __init__(), upload(), debug_info()

JSONAttachment

A convenience class for creating attachments from JSON-serializable objects. Methods __init__()

MetricSummary

Summary of a metric’s performance. Properties

str

Name of the metric.

float | int

Average metric across all examples.

str

Unit label for the metric.

int | None

Number of improvements in the metric.

int | None

Number of regressions in the metric.

float | None

Difference in metric between the current and reference experiment.

Methods __init__()

ParametersBuilder

Builder to create saved parameters in Braintrust. Properties

Any

Methods __init__(), create()

ParametersRef

Reference to saved parameters by ID and optional version. Properties

str

Project

A handle to a Braintrust project. Properties

Any

Methods __init__(), add_code_function(), add_prompt(), add_parameters(), publish()

ProjectBuilder

Creates handles to Braintrust projects. Methods create()

Prompt

A prompt object consists of prompt text, a model, and model parameters (such as temperature), which can be used to generate completions or chat messages. The prompt object supports calling .build() which uses mustache templating to build the prompt with the given formatting options and returns a plain dictionary that includes the built prompt and arguments. The dictionary can be passed as kwargs to the OpenAI client or modified as you see fit. Properties

Any

str

PromptBlockData | None

str

PromptOptions

Methods __init__(), from_prompt_data(), build()

PromptBuilder

Builder to create a prompt in Braintrust. Properties

Any

Methods __init__(), create()

ReadonlyAttachment

A readonly alternative to Attachment, which can be used for fetching already-uploaded Attachments. Properties

Any

bytes

The attachment contents. This is a lazy value that will read the attachment contents from the object store on first access.

Methods __init__(), metadata(), status()

ReadonlyExperiment

A read-only view of an experiment, initialized by passing open=True to init(). Properties

Any

str

BraintrustState

Methods __init__(), as_dataset()

RepoInfo

Information about the current HEAD of the repo. Properties

str | None

bool | None

str | None

Methods __init__()

ReporterDef

A reporter takes an evaluator and its result and returns a report. Properties

str

The name of the reporter.

Callable[[Evaluator[Input, Output, Expected], EvalResultWithSummary[Input, Output, Expected], bool, bool], EvalReport | Awaitable[EvalReport]]

A function that takes an evaluator and its result and returns a report.

Callable[[list[EvalReport], bool, bool], bool | Awaitable[bool]]

A function that takes all evaluator results and returns a boolean indicating whether the run was successful. If you return false, the braintrust eval command will exit with a non-zero status code.

Methods __init__()

RetryRequestExceptionsAdapter

An HTTP adapter that automatically retries requests on connection exceptions. Properties

Any

Methods __init__(), send()

SSEProgressEvent

A progress event that can be reported during task execution, specifically for SSE (Server-Sent Events) streams. This is a subclass of TaskProgressEvent with additional fields for SSE-specific metadata. Properties

str

ObjectReference

str

ScoreSummary

Summary of a score’s performance. Properties

str

Name of the score.

float

Average score across all examples.

int | None

Number of improvements in the score.

int | None

Number of regressions in the score.

float | None

Difference in score between the current and reference experiment.

Methods __init__()

ScorerBuilder

Builder to create a scorer in Braintrust. Properties

Any

Methods __init__(), create()

Span

A Span encapsulates logged data and metrics for a unit of work. This interface is shared by all span implementations. Properties

str

Row ID of the span.

str

Name of the span, for display purposes only.

Methods log(), log_feedback(), start_span(), export(), inject(), link(), permalink(), end(), flush(), close(), set_attributes(), set_current(), unset_current()

SpanIds

The three IDs that define a span’s position in the trace tree. Properties

str

list[str] | None

Methods __init__()

SpanImpl

Primary implementation of the Span interface. See the Span interface for full details on each method. Properties

bool

Any

str

Methods __init__(), set_attributes(), log(), log_internal(), log_feedback(), start_span(), end(), export(), inject(), link(), permalink(), close(), flush(), set_current(), unset_current()

SpanInternalOptions

Options reserved for Braintrust SDK internals. Properties

str

Identifier for the instrumentation code creating the span. Stamped into context.span_origin.instrumentation.name. Set by SDK integrations (openai-auto, anthropic-auto, etc.).

SpanScope

Scope for operating on a single span. Properties

Literal['span']

str

SyncScorerLike

Protocol for synchronous scorers that implement the callable interface. This is the most common interface and is used when no async version is available. Methods __call__()

TaskProgressEvent

Progress event that can be reported during task execution. Properties

FunctionFormat

FunctionOutputType

Literal['reasoning_delta', 'text_delta', 'json_delta', 'error', 'console', 'start', 'done', 'progress']

str

ToolBuilder

Builder to create a tool in Braintrust. Properties

Any

Methods __init__(), create()

TraceScope

Scope for operating on an entire trace. Properties

Literal['trace']

str

Integration modules

braintrust.integrations.adk

Braintrust integration for Google ADK. Import from braintrust.integrations.adk.

adk.ADKIntegration

Braintrust instrumentation for Google ADK (Agent Development Kit). Properties

Any

adk.setup_adk

Setup Braintrust integration with Google ADK. Will automatically patch Google ADK agents, runners, flows, and MCP tools for automatic tracing.

str | None

Braintrust API key.

str | None

Braintrust project ID.

str | None

Braintrust project name.

type | None

Deprecated parameter.

adk.setup_braintrust

Any

adk.wrap_agent

Manually patch an agent class for tracing.

Any

adk.wrap_runner

Manually patch a runner class for tracing.

Any

adk.wrap_flow

Manually patch a flow class for tracing.

Any

adk.wrap_mcp_tool

Manually patch an MCP tool class for tracing.

Any

braintrust.integrations.agentscope

Braintrust integration for AgentScope. Import from braintrust.integrations.agentscope.

agentscope.AgentScopeIntegration

Braintrust instrumentation for AgentScope. Requires AgentScope v1.0.0 or higher. Properties

Any

agentscope.setup_agentscope

Setup Braintrust integration with AgentScope.

str | None

braintrust.integrations.agno

Braintrust integration for Agno. Import from braintrust.integrations.agno.

agno.AgnoIntegration

Braintrust instrumentation for Agno. Properties

Any

agno.setup_agno

Setup Braintrust integration with Agno. Will automatically patch Agno agents, models, and function calls for tracing.

str | None

Braintrust API key (optional, can use env var BRAINTRUST_API_KEY)

str | None

Braintrust project ID (optional)

str | None

Braintrust project name (optional, can use env var BRAINTRUST_PROJECT)

agno.wrap_agent

Manually patch an Agent class for tracing.

Any

agno.wrap_function_call

Manually patch a FunctionCall class for tracing.

Any

agno.wrap_model

Manually patch a Model class for tracing.

Any

agno.wrap_team

Manually patch a Team class for tracing.

Any

agno.wrap_workflow

Manually patch a Workflow class for tracing.

Any

braintrust.integrations.anthropic

Import from braintrust.integrations.anthropic.

anthropic.AnthropicIntegration

Braintrust instrumentation for the Anthropic Python SDK on anthropic>=0.48.0. Properties

Any

anthropic.wrap_anthropic_client

Any

braintrust.integrations.autogen

Braintrust AutoGen integration. Import from braintrust.integrations.autogen.

autogen.AutoGenIntegration

Braintrust instrumentation for Microsoft AutoGen AgentChat. Properties

Any

autogen.setup_autogen

Setup Braintrust integration with AutoGen.

str | None

braintrust.integrations.bedrock_runtime

Public entry points for the boto3 Bedrock Runtime integration. Import from braintrust.integrations.bedrock_runtime.

bedrock_runtime.BedrockRuntimeIntegration

Braintrust instrumentation for boto3 Bedrock Runtime clients. Properties

Any

bedrock_runtime.setup_bedrock

Patch botocore client creation to auto-wrap Bedrock Runtime clients.

bedrock_runtime.wrap_bedrock

Instrument a boto3 Bedrock Runtime client instance in place.

Any

braintrust.integrations.claude_agent_sdk

Braintrust integration for Claude Agent SDK with automatic tracing. Import from braintrust.integrations.claude_agent_sdk.

claude_agent_sdk.ClaudeAgentSDKIntegration

Braintrust instrumentation for the Claude Agent SDK. Properties

Any

claude_agent_sdk.setup_claude_agent_sdk

Setup Braintrust integration with Claude Agent SDK. Will automatically patch the SDK for automatic tracing.

str | None

Braintrust API key.

str | None

Braintrust project ID.

str | None

Braintrust project name.

braintrust.integrations.cohere

Braintrust integration for the Cohere Python SDK. Import from braintrust.integrations.cohere.

cohere.CohereIntegration

Braintrust instrumentation for the Cohere Python SDK. Properties

Any

cohere.wrap_cohere

Wrap a Cohere client instance for Braintrust tracing.

Any

braintrust.integrations.crewai

Braintrust integration for CrewAI. Import from braintrust.integrations.crewai.

crewai.BraintrustCrewAIListener

CrewAI event-bus listener that maps events into Braintrust spans.

crewai.CrewAIIntegration

Braintrust instrumentation for CrewAI. Properties

Any

crewai.patch_crewai

crewai.setup_crewai

Set up Braintrust tracing for CrewAI.

str | None

Braintrust API key (optional, BRAINTRUST_API_KEY env var works too).

str | None

Braintrust project id (optional).

str | None

Braintrust project name (optional, BRAINTRUST_PROJECT env var works too).

braintrust.integrations.dspy

Braintrust integration for DSPy. Import from braintrust.integrations.dspy.

dspy.BraintrustDSpyCallback

Callback handler that logs DSPy execution traces to Braintrust. Methods __init__(), on_lm_start(), on_lm_end(), on_module_start(), on_module_end(), on_adapter_format_start(), on_adapter_format_end(), on_adapter_parse_start(), on_adapter_parse_end(), on_tool_start(), on_tool_end(), on_evaluate_start(), on_evaluate_end()

dspy.DSPyIntegration

Braintrust instrumentation for DSPy. Properties

Any

dspy.patch_dspy

Patch DSPy to automatically add Braintrust tracing callback.

braintrust.integrations.google_genai

Braintrust integration for Google GenAI. Import from braintrust.integrations.google_genai.

google_genai.GoogleGenAIIntegration

Braintrust instrumentation for the Google GenAI Python SDK. Properties

Any

google_genai.setup_genai

Setup Braintrust integration with Google GenAI.

str | None

Braintrust API key.

str | None

Braintrust project ID.

str | None

Braintrust project name.

braintrust.integrations.huggingface_hub

Braintrust integration for the HuggingFace Hub Python SDK. Import from braintrust.integrations.huggingface_hub.

huggingface_hub.HuggingFaceHubIntegration

Braintrust instrumentation for the HuggingFace Hub Python SDK. Properties

Any

huggingface_hub.wrap_huggingface_hub

Wrap an InferenceClient or AsyncInferenceClient for Braintrust tracing.

Any

braintrust.integrations.instructor

Braintrust integration for the Instructor structured-output library. Import from braintrust.integrations.instructor.

instructor.InstructorIntegration

Braintrust instrumentation for the Instructor structured-output library. Properties

Any

instructor.wrap_instructor

Instrument an instructor.Instructor / AsyncInstructor client in place.

Any

braintrust.integrations.langchain

Braintrust integration for LangChain. Import from braintrust.integrations.langchain.

langchain.BraintrustCallbackHandler

Methods __init__()

langchain.LangChainIntegration

Braintrust instrumentation for LangChain via a global callback handler. Properties

Any

langchain.set_global_handler

Any

langchain.setup_langchain

Setup Braintrust integration with LangChain.

str | None

braintrust.integrations.litellm

Braintrust LiteLLM integration. Import from braintrust.integrations.litellm.

litellm.LiteLLMIntegration

Braintrust instrumentation for the LiteLLM Python SDK. Properties

Any

litellm.patch_litellm

Patch LiteLLM to add Braintrust tracing.

litellm.wrap_litellm

Wrap a LiteLLM module to add Braintrust tracing.

Any

The litellm module or a module-like object that exposes the standard LiteLLM top-level functions.

braintrust.integrations.livekit_agents

Braintrust integration for LiveKit Agents. Import from braintrust.integrations.livekit_agents.

livekit_agents.LiveKitAgentsIntegration

Properties

Any

livekit_agents.setup_livekit_agents

livekit_agents.wrap_livekit_agents

Instrument a LiveKit Agents class or instance directly.

Any

braintrust.integrations.llamaindex

Braintrust integration for LlamaIndex. Import from braintrust.integrations.llamaindex.

llamaindex.BraintrustSpanHandler

Methods __init__()

llamaindex.LlamaIndexIntegration

Braintrust instrumentation for LlamaIndex via dispatcher handlers. Properties

Any

llamaindex.setup_llamaindex

str | None

braintrust.integrations.mistral

Braintrust integration for the Mistral Python SDK. Import from braintrust.integrations.mistral.

mistral.MistralIntegration

Braintrust instrumentation for the Mistral Python SDK. Properties

Any

mistral.wrap_mistral

Wrap a single Mistral client instance for tracing.

Any

braintrust.integrations.openai

Braintrust integration for the OpenAI Python SDK and OpenAI-compatible gateways. Import from braintrust.integrations.openai.

openai.OpenAIIntegration

Braintrust instrumentation for the OpenAI Python SDK. Properties

Any

openai.setup_openai

Setup Braintrust integration with OpenAI.

str | None

Braintrust API key (optional, can use env var BRAINTRUST_API_KEY)

str | None

Braintrust project ID (optional)

str | None

Braintrust project name (optional, can use env var BRAINTRUST_PROJECT)

openai.wrap_openai

Manually wrap an OpenAI client instance for tracing.

Any

braintrust.integrations.openai_agents

Braintrust integration for the OpenAI Agents SDK. Import from braintrust.integrations.openai_agents.

openai_agents.BraintrustTracingProcessor

Methods __init__()

openai_agents.OpenAIAgentsIntegration

Braintrust instrumentation for the OpenAI Agents SDK. Properties

Any

openai_agents.setup_openai_agents

Setup Braintrust tracing for the OpenAI Agents SDK.

str | None

braintrust.integrations.openrouter

Braintrust integration for the OpenRouter Python SDK. Import from braintrust.integrations.openrouter.

openrouter.OpenRouterIntegration

Braintrust instrumentation for the OpenRouter Python SDK. Properties

Any

openrouter.wrap_openrouter

Wrap a single OpenRouter client instance for tracing.

Any

braintrust.integrations.pydantic_ai

Braintrust integration for Pydantic AI. Import from braintrust.integrations.pydantic_ai.

pydantic_ai.PydanticAIIntegration

Braintrust instrumentation for Pydantic AI. Properties

Any

pydantic_ai.setup_pydantic_ai

Setup Braintrust integration with Pydantic AI. Will automatically patch Pydantic AI agents and direct API functions for automatic tracing.

str | None

Braintrust API key.

str | None

Braintrust project ID.

str | None

Braintrust project name.

pydantic_ai.wrap_agent

Any

pydantic_ai.wrap_model_classes

Deprecated compatibility shim for scanning currently loaded model subclasses.

pydantic_ai.wrap_model_request

Any

pydantic_ai.wrap_model_request_sync

Any

pydantic_ai.wrap_model_request_stream

Any

pydantic_ai.wrap_model_request_stream_sync

Any

braintrust.integrations.strands

Braintrust integration for Strands Agents. Import from braintrust.integrations.strands.

strands.StrandsIntegration

Braintrust instrumentation for Strands Agents. Properties

Any

strands.setup_strands

Set up Braintrust tracing for Strands Agents.

str | None

strands.wrap_strands_tracer

Manually patch a Strands Tracer class for Braintrust tracing.

Any

braintrust.integrations.temporal

Braintrust integration for Temporal workflows and activities. Import from braintrust.integrations.temporal.

temporal.BraintrustInterceptor

Braintrust interceptor for tracing Temporal workflows and activities. Properties

Any

Methods __init__(), intercept_client(), intercept_activity(), workflow_interceptor_class()

temporal.BraintrustPlugin

Braintrust plugin for Temporal that automatically configures tracing. Methods __init__()

temporal.TemporalIntegration

Braintrust instrumentation for Temporal workflows and activities. Properties

Any

temporal.setup_temporal

Set up Braintrust auto-instrumentation for Temporal.

str | None

Braintrust Python SDK

⌘I

​Requirements

​Installation

​Functions

​Eval

​EvalAsync

​Reporter

​auto_instrument

​current_experiment

​current_logger

​current_span

​extract_trace_context

​flush

​get_prompt_versions

​get_span_parent_object

​init

​init_dataset

​init_experiment

​init_function

​init_logger

​inject_trace_context

​invoke

​load_parameters

​load_prompt

​load_prompt_async

​log

​login

​parent_context

​parse_stream

​patch_openai

​permalink

​register_otel_flush

​run_evaluator

​set_http_adapter

​set_masking_function

​set_thread_pool_max_workers

​span_components_to_object_id

​start_span

​summarize

​traced

​update_span

​Classes

​AsyncScorerLike

​Attachment

​BaseExperiment

​BraintrustConsoleChunk

​BraintrustErrorChunk

​BraintrustInvokeError

​BraintrustJsonChunk

​BraintrustProgressChunk

​BraintrustStream

​BraintrustTextChunk

​ClassifierBuilder

​CodeFunction

​CodePrompt

​DataSummary

​Dataset

​DatasetRef

​DatasetSummary

​EvalCase

​EvalHooks

​EvalResult

​EvalScorerArgs

​Evaluator

​Experiment

​ExperimentSummary

​ExternalAttachment

​JSONAttachment

​MetricSummary

​ParametersBuilder

​ParametersRef

​Project

​ProjectBuilder

​Prompt

​PromptBuilder

​ReadonlyAttachment

​ReadonlyExperiment

​RepoInfo

​ReporterDef

​RetryRequestExceptionsAdapter

​SSEProgressEvent

Requirements

Installation

Functions

Eval

EvalAsync

Reporter

auto_instrument

current_experiment

current_logger

current_span

extract_trace_context

flush

get_prompt_versions

get_span_parent_object

init

init_dataset

init_experiment

init_function

init_logger

inject_trace_context

invoke

load_parameters

load_prompt

load_prompt_async

log

login

parent_context

parse_stream

patch_openai

permalink

register_otel_flush

run_evaluator

set_http_adapter

set_masking_function

set_thread_pool_max_workers

span_components_to_object_id

start_span

summarize

traced

update_span

Classes

AsyncScorerLike

Attachment

BaseExperiment

BraintrustConsoleChunk

BraintrustErrorChunk

BraintrustInvokeError

BraintrustJsonChunk

BraintrustProgressChunk

BraintrustStream

BraintrustTextChunk

ClassifierBuilder

CodeFunction

CodePrompt

DataSummary

Dataset

DatasetRef

DatasetSummary

EvalCase

EvalHooks

EvalResult

EvalScorerArgs

Evaluator

Experiment

ExperimentSummary

ExternalAttachment

JSONAttachment

MetricSummary

ParametersBuilder

ParametersRef

Project

ProjectBuilder

Prompt

PromptBuilder

ReadonlyAttachment

ReadonlyExperiment

RepoInfo

ReporterDef

RetryRequestExceptionsAdapter

SSEProgressEvent