BenchmarkConfigΒΆ

class BenchmarkConfig ( VeekshaCommand )

Run a benchmark against an LLM inference server.

Fields:

output_dirstr = "benchmark_output"

Base directory for all benchmark outputs (traces, metrics, logs)

seedint = 42

Seed for the random number generator.

session_generatorBaseSessionGeneratorConfig = SyntheticSessionGeneratorConfig

The session generator configuration for the benchmark.

Available types:

traffic_schedulerBaseTrafficConfig = RateTrafficConfig

The traffic scheduler configuration for the benchmark.

Available types:

evaluatorslist [ BaseEvaluatorConfig ] = [<PerformanceEvaluatorConfig>]

List of evaluators to run.

Available types:

clientBaseClientConfig = OpenAIChatCompletionsClientConfig

The client configuration for the benchmark.

Available types:

runtimeRuntimeConfig = RuntimeConfig

The runtime configuration for the benchmark.

trace_recorderTraceRecorderConfig = TraceRecorderConfig

Trace recorder configuration. Records dispatched requests (unlike the evaluator, which records them after completion).

serverOptional [ BaseServerConfig ] = None

Server configuration for managed servers. If set, it produces endpoint.

Available types:

endpointOptional [ EndpointConfig ] = None

Client-facing endpoint contract used to overwrite client.model, client.api_key and client.api_base.

wandbWandbConfig = WandbConfig

Weights & Biases logging configuration.