LMEvalAccuracyEvaluatorConfig

class LMEvalAccuracyEvaluatorConfig ( BaseEvaluatorConfig )

Configuration for lm-eval accuracy evaluation (task-specific correctness).

Polymorphic Type:

type: accuracy_lmeval

All BaseEvaluatorConfig types:

• performance: PerformanceEvaluatorConfig

• accuracy_lmeval: LMEvalAccuracyEvaluatorConfig

• audio_quality: AudioQualityEvaluatorConfig

Fields:

target_channelslist = ['text']

List of ChannelModality values to evaluate.

sloslist [ BaseSloConfig ] = []

Accuracy evaluators do not use performance SLOs.

Available types:

• constant: ConstantSloConfig

stream_metricsbool = False

Accuracy evaluation does not stream incremental metrics.

stream_metrics_intervalfloat = 5.0

Interval for streaming metrics in seconds

bootstrap_itersint = 100000

Bootstrap iterations for confidence intervals