LLMSimilarityScorer

classDiagram ScorerBase <|-- LLMSimilarityScorer

class council.scorers.LLMSimilarityScorer(llm: LLMBase, expected: str)

Bases: ScorerBase

Using an LLM to compute a similarity score between two messages.

__init__(llm: LLMBase, expected: str)

Initialize a new instance

Parameters:

• llm (LLMBase) – the LLM to be used

• expected (str) – the expected text message

score(message: ChatMessage) → float

Score the given message

Parameters:

message (ChatMessage) – the message to be scored

Returns:

similarity score. The greater the value to higher the similarity

Raises:

SimilarityScorerException – an unexpected error occurs

to_dict() → Dict[str, Any]

Serialize the instance into a dictionary. May need to be overriden in derived classes