BenchmarkConfig
===============

**class** ``BenchmarkConfig`` **(** ``VeekshaCommand`` **)**

    Run a benchmark against an LLM inference server.

**Fields:**

``output_dir`` : *str* = ``"benchmark_output"``
    Base directory for all benchmark outputs (traces, metrics, logs)

``seed`` : *int* = ``42``
    Seed for the random number generator.

``session_generator`` : :doc:`BaseSessionGeneratorConfig` = :doc:`SyntheticSessionGeneratorConfig`
    The session generator configuration for the benchmark.

    Available types:

    - ``synthetic``: :doc:`SyntheticSessionGeneratorConfig`
    - ``lmeval``: :doc:`LmevalSessionGeneratorConfig`
    - ``trace``: :doc:`TraceSessionGeneratorConfig`

``traffic_scheduler`` : :doc:`BaseTrafficConfig` = :doc:`RateTrafficConfig`
    The traffic scheduler configuration for the benchmark.

    Available types:

    - ``rate``: :doc:`RateTrafficConfig`
    - ``concurrent``: :doc:`ConcurrentTrafficConfig`
    - ``sequential_launch``: :doc:`SequentialLaunchTrafficConfig`

``evaluators`` : *list* [ :doc:`BaseEvaluatorConfig` ] = ``[<PerformanceEvaluatorConfig>]``
    List of evaluators to run.

    Available types:

    - ``performance``: :doc:`PerformanceEvaluatorConfig`
    - ``accuracy_lmeval``: :doc:`LMEvalAccuracyEvaluatorConfig`

``client`` : :doc:`BaseClientConfig` = :doc:`OpenAIChatCompletionsClientConfig`
    The client configuration for the benchmark.

    Available types:

    - ``openai_chat_completions``: :doc:`OpenAIChatCompletionsClientConfig`
    - ``openai_completions``: :doc:`OpenAICompletionsClientConfig`
    - ``openai_router``: :doc:`OpenAIRouterClientConfig`

``runtime`` : :doc:`RuntimeConfig` = :doc:`RuntimeConfig`
    The runtime configuration for the benchmark.

``trace_recorder`` : :doc:`TraceRecorderConfig` = :doc:`TraceRecorderConfig`
    Trace recorder configuration. Records dispatched requests (unlike the evaluator, which records them after completion).

``server`` : *Optional* [ :doc:`BaseServerConfig` ] = ``None``
    Server configuration for managed servers. If set, client.model, client.api_key and client.api_base will be overwritten.

    Available types:

    - ``vllm``: :doc:`VllmServerConfig`
    - ``vajra``: :doc:`VajraServerConfig`
    - ``sglang``: :doc:`SglangServerConfig`

``wandb`` : :doc:`WandbConfig` = :doc:`WandbConfig`
    Weights & Biases logging configuration.