PyPI page
Home page
Author:
Adarsh Dwivedi
Summary:
A drop-in semantic cache for LLM APIs: match requests by meaning, serve repeats instantly, and route across OpenAI, Gemini, Groq, Claude, Ollama and Bedrock.
Latest version:
0.3.0
Required dependencies:
fastapi
|
fastembed
|
httpx
|
numpy
|
prometheus-client
|
pydantic-settings
|
structlog
|
typer
|
uvicorn
Optional dependencies:
boto3
|
botocore
|
cachellm-proxy
|
datasets
|
langfuse
|
opentelemetry-exporter-otlp-proto-http
|
opentelemetry-instrumentation-botocore
|
opentelemetry-instrumentation-fastapi
|
opentelemetry-instrumentation-httpx
|
opentelemetry-instrumentation-redis
|
opentelemetry-sdk
|
pandas
|
redis
|
redisvl
Downloads last day:
8
Downloads last week:
56
Downloads last month:
602
7-day smoothing