Proposal: Cache-optimized routing (#314)

Goal: Route requests to take advantage of prefix caching. Proposal: Use hashing to define preferred backends for given requests. This avoids the need to keep track of the state of backend caches. Reference: https://research.google/blog/consistent-hashing-with-bounded-loads/
substratusai · Dec 3, 2024 · 8eaa1db · 8eaa1db
1 parent 2c303b9
commit 8eaa1db
Showing 1 changed file with 0 additions and 0 deletions.
diff --git a/proposals/diagrams/cache-optimized-routing.excalidraw.png b/proposals/diagrams/cache-optimized-routing.excalidraw.png