LLMEvaluator

classDiagram EvaluatorBase <|-- LLMEvaluator

class council.evaluators.LLMEvaluator(llm: LLMBase)

Bases: EvaluatorBase

Evaluator using an LLM to evaluate chain responses.

__init__(llm: LLMBase)

Build a new LLMEvaluator.

Parameters:

llm – model to use for the evaluation.

execute(context: AgentContext) → List[ScoredChatMessage]

Executes the evaluator on the agent’s context within the given budget.

Parameters:

context (AgentContext) – The context for executing the evaluator.

Returns:

A list of scored agent messages resulting from the evaluation.

Return type:

List[ScoredChatMessage]

Raises:

None –

property llm: LLMBase

the LLM used by the evaluator

render_as_dict() → Dict[str, Any]

returns the graph of operation as a dictionary

render_as_json() → str

returns the graph of operation as a JSON string