BenchmarkConfig

class BenchmarkConfig ( VeekshaCommand )

Run a benchmark against an LLM inference server.

Fields:

output_dirstr = "benchmark_output"

Base directory for all benchmark outputs (traces, metrics, logs)

seedint = 42

Seed for the random number generator.

session_generatorBaseSessionGeneratorConfig = SyntheticSessionGeneratorConfig

The session generator configuration for the benchmark.

Available types:

• synthetic: SyntheticSessionGeneratorConfig

• lmeval: LmevalSessionGeneratorConfig

• trace: TraceSessionGeneratorConfig

traffic_schedulerBaseTrafficConfig = RateTrafficConfig

The traffic scheduler configuration for the benchmark.

Available types:

• rate: RateTrafficConfig

• concurrent: ConcurrentTrafficConfig

• sequential_launch: SequentialLaunchTrafficConfig

evaluatorslist [ BaseEvaluatorConfig ] = [<PerformanceEvaluatorConfig>]

List of evaluators to run.

Available types:

• performance: PerformanceEvaluatorConfig

• accuracy_lmeval: LMEvalAccuracyEvaluatorConfig

clientBaseClientConfig = OpenAIChatCompletionsClientConfig

The client configuration for the benchmark.

Available types:

• openai_chat_completions: OpenAIChatCompletionsClientConfig

• openai_completions: OpenAICompletionsClientConfig

• openai_router: OpenAIRouterClientConfig

runtimeRuntimeConfig = RuntimeConfig

The runtime configuration for the benchmark.

trace_recorderTraceRecorderConfig = TraceRecorderConfig

Trace recorder configuration. Records dispatched requests (unlike the evaluator, which records them after completion).

serverOptional [ BaseServerConfig ] = None

Server configuration for managed servers. If set, client.model, client.api_key and client.api_base will be overwritten.

Available types:

• vllm: VllmServerConfig

• vajra: VajraServerConfig

• sglang: SglangServerConfig

wandbWandbConfig = WandbConfig

Weights & Biases logging configuration.