ADK.Eval.Case (ADK v0.0.1-alpha.1)

Defines a single evaluation test case.

A case specifies an input message to send to an agent, a list of scorers to evaluate the response, and optional metadata.

Example

ADK.Eval.Case.new(
  name: "greeting",
  input: "Hello!",
  scorers: [
    {ADK.Eval.Scorer.Contains, text: "hello", case_sensitive: false},
    {ADK.Eval.Scorer.ResponseLength, min: 5, max: 200}
  ],
  metadata: %{category: "basic"}
)

Summary

Types

scorer_config()

t()

Functions

new(opts)

Create a new eval case.

Types

scorer_config()

@type scorer_config() :: {module(), keyword()}

t()

@type t() :: %ADK.Eval.Case{
  input: String.t(),
  metadata: map(),
  name: String.t(),
  scorers: [scorer_config()]
}

Functions

new(opts)

@spec new(keyword()) :: t()

Create a new eval case.