LLMMiddleware#

class council.llm.LLMMiddleware(*args, **kwargs)[source]#

Bases: Protocol

Protocol for defining LLM middleware.

Middleware can intercept and modify requests and responses between the client and the LLM, introducing custom logic.

LLMMiddlewareChain#

class council.llm.LLMMiddlewareChain(llm: LLMBase, middlewares: Sequence[LLMMiddleware] | None = None)[source]#

Bases: object

Manages a chain of LLM middlewares and executes requests through them.

add_middleware(middleware: LLMMiddleware) → None[source]#: Add middleware to a chain.

execute(request: LLMRequest) → LLMResponse[source]#: Execute middleware chain.

class council.llm.LLMLoggingMiddleware(log_consumptions: bool = False)[source]#

Bases: object

Middleware for logging LLM requests, responses and consumptions.

class council.llm.LLMFileLoggingMiddleware(log_file: str, component_name: str, log_consumptions: bool = False)[source]#

Bases: object

Middleware for logging LLM requests, responses and consumptions into a file.

__init__(log_file: str, component_name: str, log_consumptions: bool = False) → None[source]#: Initialize the middleware with the path to the log_file, component name and whether to log consumptions

class council.llm.LLMRetryMiddleware(retries: int, delay: float, exception_to_check: type[Exception] | None = None)[source]#

Bases: object

Middleware for implementing retry logic for LLM requests.

Attempts to retry failed requests a specified number of times with a delay between attempts.

class council.llm.LLMCachingMiddleware(ttl: float = 300.0, cache_limit_size: int = 10)[source]#

Bases: object

Middleware that caches LLM responses to avoid duplicate calls.

__init__(ttl: float = 300.0, cache_limit_size: int = 10) → None[source]#

Initialize the caching middleware.

Parameters:

ttl – Sliding window time-to-live in seconds for cache entries (default: 5 mins)
cache_limit_size – Cache limit size in cached entries (default: 10)

static get_hash(request: LLMRequest, configuration: T_Configuration) → str[source]#: Convert the request and LLM configuration to a hash with hashlib.sha256.

class council.llm.LLMRequest(context: LLMContext, messages: Sequence[LLMMessage], **kwargs: Any)[source]#

Bases: object

static default(messages: Sequence[LLMMessage], **kwargs: Any) → LLMRequest[source]#: Creates a default LLMRequest with an empty context.

class council.llm.LLMResponse(request: LLMRequest, result: LLMResult | None, duration: float)[source]#

Bases: object

static empty(request: LLMRequest) → LLMResponse[source]#: Creates an empty LLMResponse for a given request.