LMEvalAccuracyEvaluatorConfig

class LMEvalAccuracyEvaluatorConfig ( BaseEvaluatorConfig )

Configuration for lm-eval accuracy evaluation (task-specific correctness).

Polymorphic Type:

type: accuracy_lmeval

All BaseEvaluatorConfig types:

• performance: PerformanceEvaluatorConfig

• accuracy_lmeval: LMEvalAccuracyEvaluatorConfig

Fields:

target_channelslist = ['text']

List of ChannelModality values to evaluate.

sloslist [ BaseSloConfig ] = [<ConstantSloConfig>]

List of SLO definitions to evaluate against request-level metrics.

Available types:

• constant: ConstantSloConfig

stream_metricsbool = True

Enable real-time metric streaming

stream_metrics_intervalfloat = 5.0

Interval for streaming metrics in seconds

bootstrap_itersint = 100000

Bootstrap iterations for confidence intervals