Letting varnish always serve out cached content, and update its cache in the background

published Jan 31, 2013 05:35 by admin ( last modified Apr 12, 2014 02:29 )

Update 2014-04-12

This function now seems to be in the new varnish 4. From the release notes:

Background (re)fetch of expired objects. On a cache miss where a stale copy is available, serve the client the stale copy while fetching an updated copy from the backend in the background.

If you cache content in Varnish, it serves out pages really quickly. But when the cached object expires, the next visitor has to wait for the slow back end, and then the object gets cached once again.

What about if I'd like Varnish to always serve out cached content even when it has expired and then silently - in the background - fetch a new copy for its cache? In that way no visitor ever has to wait for the back end.

Update 2013-10-19:

See the link near the end of this blog post for what seems to be a working, but complex solution (or better, just update to varnish 4.x).

This holy grail feature is called "stale-while-revalidate". It does not currently seem possible in straightforward way, from the linked documentation (my boldface):

With this exercise, you should see that as long as the content is within the regular TTL, there is no difference. Once the TTL expires, the first client that asks for the content should be stuck for 15 seconds, while the second client should get the graced copy.

However, you can trigger an external http client when the time comes close:

Varnish trick: Serve stale content while refetching | Blog of Lasse Karstensen

...that is kludgy, though.