Evaluation Studio
Create and run evaluation suites to benchmark and compare AI model outputs.
Create evaluation suite
Creates a new evaluation suite grouping a set of test cases and scoring criteria for benchmarking AI model outputs. Suites are used in the AI Studio to compare prompt template versions or to validate governance policy impact on response quality.
Update evaluation suite
Updates the name, description, or scoring criteria of an existing evaluation suite.
Delete evaluation suite
Permanently removes an evaluation suite and all associated run records.
Run AI evaluation suite
Triggers execution of an evaluation suite, running all test cases against the configured AI model and scoring the outputs according to the suite's criteria. Results are stored asynchronously; poll the /evals/{id} endpoint to check completion. Use to validate prompt changes or measure regression after a model update.
Read AI evaluation result
Retrieves the current status and results of an evaluation run. Poll this endpoint after triggering a run via POST /evals until status transitions to 'completed' or 'failed'. The score property contains the weighted aggregate score once the run completes.