- Amazon ElastiCache for Valkey now supports fully managed distributed KV caching for self-hosted LLM inference.
- The capability reduces inference costs by up to 40% and time to first token by up to 80% using cache.i8ge.48xlarge nodes and RDMA over EFA.
- It integrates with inference engines like vLLM, SGLang, and TensorRT-LLM via the LMCache connector across 13 AWS Regions.