Whilst working with Magento 2 there are a number of things that crop up frequently and take time to resolve. (If you're anything like me you remember 'fix' from last time it got you, right after working out the fix all over again.)
In this guide we are going to look at two common pitfalls/gotchas when working with Varnish and Magento 2. This guide is applicable for all Magento 2 sites, whether you are using MDOQ managed Magento hosting or not.
Enable Full Page Cache!
The first gotcha occurs when you disable full page cache, when Varnish already has items cached.
In this scenario no mater how many times we run
php bin/magento cache:flush
Varnish still has the item in cache. No matter what Magento command you run, you will not be able to clear these items using Magento.
Why?
This is because when you disable full page cache, Magento will not send purge requests to Varnish (Because it's disabled). So any items that are in the Varnish before then will persist. Because Varnish sits much further up the stack and has no knowledge of Magento (it will never ask Magento "Hey am I enabled?") it is behaving as intended.
Solution
If you want to disable full page cache, just flush it before you disable it.
php bin/magento cache:flush && php bin/magento cache:disable full_page
If it's already disabled, enable it flush cache and then disable it.
php bin/magento cache:enable full_page && \
php bin/magento cache:flush && \
php bin/magento cache:disable full_page
Ban Everything!
Have you ever put a die in a script file, or caused an error/exception only to find that the resulting page is now completely stuck in Varnish and no matter what you do you cannot clear it from Varnish (without restarting the varnish service or using varnishadm) ?
Why?
When Magento caches something in Varnish it uses a header X-Magento-Tags-Pattern so when you run cache flush, Magento sends a PURGE request to Varnish containing a header like this:
X-Magento-Tags-Pattern: .*
This is then processed by Varnish using the rules in the vcl file.
if (req.http.X-Magento-Tags-Pattern) {
ban("obj.http.X-Magento-Tags ~ " + req.http.X-Magento-Tags-Pattern);
}
Which translates to "Varnish ban everything that has the tag X-Magento-Tags-Pattern with a value matching anything", which sounds okay.
However when your die get's hit, or your exception happens, it may have happened before Magento has had chance to set the X-Magento-Tags-Pattern header, which means your page has ended up in Varnish with no X-Magento-Tags-Pattern header at all. Meaning it will never be matched.
Solution
The best way I have found to solve this, that is also safe for production is to update the vcl file to the following
if (req.http.X-Magento-Tags-Pattern) {
if (req.http.X-Magento-Tags-Pattern == ".*") {
ban("obj.status != 0");
}
ban("obj.http.X-Magento-Tags ~ " + req.http.X-Magento-Tags-Pattern);
}
This will mean when Magento sends the purge request if the value of the header is ".*" then Varnish will ban everything. (See here for a really good guide on how/why the obj.status != 0 is a good way to clear everything)
N.B under normal operation Magento will clear specific tags from Varnish. i.e when saving a product it will clear all items that have that products cache tag. In which case the value will not be ".*" which will means our new logic will not be hit, which is a good thing!