LmevalSessionGeneratorConfig

class LmevalSessionGeneratorConfig ( BaseSessionGeneratorConfig )

Polymorphic Type:

type: lmeval

All BaseSessionGeneratorConfig types:

• synthetic: SyntheticSessionGeneratorConfig

• lmeval: LmevalSessionGeneratorConfig

• trace: TraceSessionGeneratorConfig

Fields:

taskslist [ str ] = ['hellaswag']

The lm-eval tasks to evaluate the model on.

num_fewshotint = 1

The number of fewshot examples to use for the tasks.