new feature: Cache API

[nilslice]

Starting a discussion thread here for beginning work on the Cache API. Docs here: https://developers.cloudflare.com/workers/learning/how-the-cache-works

[nilslice]

likely needs updates in the RequestInit object, as well as a new FFI for the caches as seen in the following JS example:

async function handleRequest(event) {
  const request = event.request
  const cacheUrl = new URL(request.url)

  // Construct the cache key from the cache URL
  const cacheKey = new Request(cacheUrl.toString(), request)
  const cache = caches.default

  // Check whether the value is already available in the cache
  // if not, you will need to fetch it from origin, and store it in the cache
  // for future access
  let response = await cache.match(cacheKey)

  if (!response) {
    // If not in cache, get it from origin
    response = await fetch(request)

    // Must use Response constructor to inherit all of response's fields
    response = new Response(response.body, response)

    // Cache API respects Cache-Control headers. Setting s-max-age to 10
    // will limit the response to be in cache for 10 seconds max

    // Any changes made to the response here will be reflected in the cached value
    response.headers.append("Cache-Control", "s-maxage=10")

    // Store the fetched response as cacheKey
    // Use waitUntil so you can return the response without blocking on
    // writing to cache
    event.waitUntil(cache.put(cacheKey, response.clone()))
  }
  return response
}

addEventListener("fetch", event => {
  try {
    const request = event.request
    return event.respondWith(handleRequest(event))
  } catch (e) {
    return event.respondWith(new Response("Error thrown " + e.message))
  }
})

[jsievenpiper]

I think the above sample mostly well-covers our use cases for the Cache API as well. I'd say of particular interest are the pre- and post- cache operations with regards to how (I think) Response objects need to be prepared for the cache. Even some simple ergonomics there go a long way (and I think the comments in the JS snippet above point at that, at least indirectly). Our flow looks something like:

Origin Service Pool -> Load Balancer w/ Affinity -> Cloudflare

The load balancer operates by applying cookies that help keep clients attached to particular instances to keep data warm, but means that by default, the Cache API won't take our Responses. So our flow looks sort of like:

// horrible rust pseudo-code

// Preprocessing
let response = fetch(request).await?;
let clean_response_headers = response.headers.iter().filter(|(key,_)| !["Set-Cookie", "Vary"].contains(key)).collect();

// Cache
cache.put(cache_key, Response::from(&response, clean_response_headers));

// Post-processing
let outgoing_response = Response::from(&response, other_headers_like_cache_control);
event.respond_with(outgoing_response);

So much like the JS example, as long as we haven't lost any of the easy cloning operations on the workers-rs Response, that would totally support our use cases. I imagine header manipulation is somewhat common, so extension traits / projected views that assist with that would be icing on the cake, but easily implemented otherwise. To add some color to that last comment, I could imagine implementing an extension trait as part of our project to make the above flow look something like:

cache.put(cache_key, response.excluding_headers(&["Set-Cookie", "Vary"]))
event.respond_with(response.including_headers(&[("Cache-Control", "s-maxage=10")]);

But that may be overly-specific to a particular use case / might not fit everyone's patterns and I totally get that.

[nilslice]

Ok great, thanks, @jsievenpiper. Both the JS example and your rust sketch bring up another implementation that needs to be integrated: FetchEvent. The linked documentation is a great starting point. We may or may not use all of those definitions depending on how the API & implementation are designed.

Beyond the binding, we need to find somewhere to put it in the API. Currently, the worker crate drops you into a entrypoint that only has access to the Request and Env.. I could see the FetchEvent being a field on the Request, inverting the actual relationship. Would need to think a bit more about how this could impact things operationally..

[praveenperera]

Is there an update on this feature?

EDIT: apparently its ready: #182 awesome