Class: Braintrust::Models::ExperimentEvent

Inherits:
BaseModel
  • Object
show all
Defined in:
lib/braintrust/models/experiment_event.rb

Defined Under Namespace

Classes: Context, Metrics, SpanAttributes

Instance Attribute Summary collapse

Method Summary

Methods inherited from BaseModel

#[], #initialize, #inspect, #to_h, #to_s

Constructor Details

This class inherits a constructor from Braintrust::BaseModel

Instance Attribute Details

#_xact_id ⇒ String

The transaction id of an event is unique to the network operation that processed the event insertion. Transaction ids are monotonically increasing over time and can be used to retrieve a versioned snapshot of the experiment (see the version parameter)

Returns:

  • (String) 


14
 
# File 'lib/braintrust/models/experiment_event.rb', line 14

required :_xact_id, String

#context ⇒ Braintrust::Models::ExperimentEvent::Context

Context is additional information about the code that produced the experiment event. It is essentially the textual counterpart to metrics. Use the caller_* attributes to track the location in code which produced the experiment event

Returns:



44
 
# File 'lib/braintrust/models/experiment_event.rb', line 44

optional :context, -> { Braintrust::Models::ExperimentEvent::Context }

#created ⇒ DateTime

The timestamp the experiment event was created

Returns:

  • (DateTime) 


19
 
# File 'lib/braintrust/models/experiment_event.rb', line 19

required :created, DateTime

#dataset_record_id ⇒ String

If the experiment is associated to a dataset, this is the event-level dataset id this experiment event is tied to

Returns:

  • (String) 


49
 
# File 'lib/braintrust/models/experiment_event.rb', line 49

optional :dataset_record_id, String

#error ⇒ Object

The error that occurred, if any.

Returns:

  • (Object) 


54
 
# File 'lib/braintrust/models/experiment_event.rb', line 54

optional :error, Braintrust::Unknown

#expected ⇒ Object

The ground truth value (an arbitrary, JSON serializable object) that you'd compare to output to determine if your output value is correct or not. Braintrust currently does not compare output to expected for you, since there are so many different ways to do that correctly. Instead, these values are just used to help you navigate your experiments while digging into analyses. However, we may later use these values to re-score outputs or fine-tune your models

Returns:

  • (Object) 


59
 
# File 'lib/braintrust/models/experiment_event.rb', line 59

optional :expected, Braintrust::Unknown

#experiment_id ⇒ String

Unique identifier for the experiment

Returns:

  • (String) 


24
 
# File 'lib/braintrust/models/experiment_event.rb', line 24

required :experiment_id, String

#id ⇒ String

A unique identifier for the experiment event. If you don't provide one, BrainTrust will generate one for you

Returns:

  • (String) 


9
 
# File 'lib/braintrust/models/experiment_event.rb', line 9

required :id, String

#input ⇒ Object

The arguments that uniquely define a test case (an arbitrary, JSON serializable object). Later on, Braintrust will use the input to know whether two test cases are the same between experiments, so they should not contain experiment-specific state. A simple rule of thumb is that if you run the same experiment twice, the input should be identical

Returns:

  • (Object) 


64
 
# File 'lib/braintrust/models/experiment_event.rb', line 64

optional :input, Braintrust::Unknown

#metadata ⇒ Hash

A dictionary with additional data about the test example, model outputs, or just about anything else that's relevant, that you can use to help find and analyze examples later. For example, you could log the prompt, example's id, or anything else that would be useful to slice/dice later. The values in metadata can be any JSON-serializable type, but its keys must be strings

Returns:

  • (Hash) 


69
 
# File 'lib/braintrust/models/experiment_event.rb', line 69

optional :metadata, Hash

#metrics ⇒ Braintrust::Models::ExperimentEvent::Metrics

Metrics are numerical measurements tracking the execution of the code that produced the experiment event. Use "start" and "end" to track the time span over which the experiment event was produced

Returns:



74
 
# File 'lib/braintrust/models/experiment_event.rb', line 74

optional :metrics, -> { Braintrust::Models::ExperimentEvent::Metrics }

#output ⇒ Object

The output of your application, including post-processing (an arbitrary, JSON serializable object), that allows you to determine whether the result is correct or not. For example, in an app that generates SQL queries, the output should be the result of the SQL query generated by the model, not the query itself, because there may be multiple valid queries that answer a single question

Returns:

  • (Object) 


79
 
# File 'lib/braintrust/models/experiment_event.rb', line 79

optional :output, Braintrust::Unknown

#project_id ⇒ String

Unique identifier for the project that the experiment belongs under

Returns:

  • (String) 


29
 
# File 'lib/braintrust/models/experiment_event.rb', line 29

required :project_id, String

#root_span_id ⇒ String

The span_id of the root of the trace this experiment event belongs to

Returns:

  • (String) 


34
 
# File 'lib/braintrust/models/experiment_event.rb', line 34

required :root_span_id, String

#scores ⇒ Hash

A dictionary of numeric values (between 0 and 1) to log. The scores should give you a variety of signals that help you determine how accurate the outputs are compared to what you expect and diagnose failures. For example, a summarization app might have one score that tells you how accurate the summary is, and another that measures the word similarity between the generated and grouth truth summary. The word similarity score could help you determine whether the summarization was covering similar concepts or not. You can use these scores to help you sort, filter, and compare experiments

Returns:

  • (Hash) 


84
 
# File 'lib/braintrust/models/experiment_event.rb', line 84

optional :scores, Hash

#span_attributes ⇒ Braintrust::Models::ExperimentEvent::SpanAttributes

Human-identifying attributes of the span, such as name, type, etc.

Returns:



89
 
# File 'lib/braintrust/models/experiment_event.rb', line 89

optional :span_attributes, -> { Braintrust::Models::ExperimentEvent::SpanAttributes }

#span_id ⇒ String

A unique identifier used to link different experiment events together as part of a full trace. See the tracing guide for full details on tracing

Returns:

  • (String) 


39
 
# File 'lib/braintrust/models/experiment_event.rb', line 39

required :span_id, String

#span_parents ⇒ Array<String>

An array of the parent span_ids of this experiment event. This should be empty for the root span of a trace, and should most often contain just one parent element for subspans

Returns:

  • (Array<String>) 


94
 
# File 'lib/braintrust/models/experiment_event.rb', line 94

optional :span_parents, Braintrust::ArrayOf.new(String)

#tags ⇒ Array<String>

A list of tags to log

Returns:

  • (Array<String>) 


99
 
# File 'lib/braintrust/models/experiment_event.rb', line 99

optional :tags, Braintrust::ArrayOf.new(String)