What's the deal with IPS's caching rules anyway?

[Linguica]

I can't really figure out what rules IPS uses for leveraging browser caching. More specifically, I don't get what's going on with the generated CSS / JS files.

Since the files are recompiled to disk with a new filename every time they are changed, it seems like the software should have very aggressive caching rules to prevent a user from redownloading them. However, just a simple F5 on a page shows that (almost) all the CSS and JS files get redownloaded *anyway*. (This is with a basically-default install, no custom htaccess rules or anything like that.)

I say "almost" because there is one exception, which makes the whole thing more baffling.

Request for the generated forums_responsive.css:

GET /uploads/css_built_2/76e62c573090645fb99a15a363d8620e_forums_responsive.css.2cd236d233926e4b6296e298e41bc8e9.css?v=ceae351820 HTTP/1.1
Host: dev.doomworld.com
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:51.0) Gecko/20100101 Firefox/51.0
Accept: text/css,*/*;q=0.1
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br
Referer: https://dev.doomworld.com/
Cookie: (stuff)
Authorization: Basic ZmFydDpmYXJ0
Connection: keep-alive
If-Modified-Since: Wed, 01 Mar 2017 02:18:32 GMT
If-None-Match: "b49-549a1edd44ec3-gzip"
Cache-Control: max-age=0

Response:

HTTP/1.1 200 OK
Date: Wed, 01 Mar 2017 19:46:21 GMT
Server: Apache/2.4.18 (Ubuntu)
Last-Modified: Wed, 01 Mar 2017 02:18:32 GMT
Etag: "b49-549a1edd44ec3-gzip"
Accept-Ranges: bytes
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Length: 687
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Content-Type: text/css

So as you can see, for a simple page reload, the browser requests the generated forums_responsive.css and it returns with a HTTP 200.

But then for the next file:

GET /uploads/css_built_2/258adbb6e4f3e83cd3b355f84e3fa002_custom.css.1691709a23ba75441ca3725242b743f8.css?v=ceae351820 HTTP/1.1
Host: dev.doomworld.com
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:51.0) Gecko/20100101 Firefox/51.0
Accept: text/css,*/*;q=0.1
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br
Referer: https://dev.doomworld.com/
Cookie: (etc)
Authorization: Basic ZmFydDpmYXJ0
Connection: keep-alive
If-Modified-Since: Wed, 01 Mar 2017 17:55:40 GMT
If-None-Match: "1b-549af054a16a1"
Cache-Control: max-age=0

HTTP/1.1 304 Not Modified
Date: Wed, 01 Mar 2017 19:46:20 GMT
Server: Apache/2.4.18 (Ubuntu)
Connection: Keep-Alive
Keep-Alive: timeout=5, max=100
Etag: "1b-549af054a16a1"

For the generated custom.css, the server (properly, really) replies with a 304 and doesn't bother sending the file again.

What's going on here? Why is the software treating these two autogenerated CSS files differently? The only real difference I can see is that the forums_responsive.css file has "-gzip" appended to its etag and the custom.css does not. But even then, the etag response matches the If-None-Match request, so why is it sending it anyway?

I could override this with a custom .htaccess that unsets the etag and adds a long-lasting "Expires:" header, but I don't really like messing with custom htaccess cache settings since it's easy to screw things up. I would really prefer to know how IPS handles browser caching and why it seems like most (but not all) of the CSS / JS files don't seem to leverage it properly.

[sudo]

Its your server doing it not IPB. You have to do expires etc with Apache/Nginx rules so I guess you have some sort of rule in your main apache conf causing the confusion.

IPB could only do it if it was served via a php file and then they would have the option to add headers but that would add a lot of load for no gain when most should do this via a web server config.

[Charles]

Yes, as ZeroHour indicated, all we do is write the CSS/JS to disk. It is up to your server or CDN to set proper cache headers for them.

[Linguica]

Well, I did some more research and it turns out it's an Apache bug: https://bz.apache.org/bugzilla/show_bug.cgi?id=45023

Long story short, on Apache 2.2 and 2.4, the mod_deflate module appends a "-gzip" to the file etag, but then when checking a request, the server doesn't check for that same etag properly. This has been known since at least 2008 and never properly fixed in the precompiled distributions.

Short of switching to Apache 2.5 or recompiling Apache yourself, turns out you can add this to your .htaccess (sigh) to basically patch it:

<IfModule mod_headers.c>
 <FilesMatch "\.(?i:js|css)$">
  RequestHeader  edit "If-None-Match" "^\"(.*)-gzip\"$" "\"$1\""
  Header  edit "ETag" "^\"(.*[^g][^z][^i][^p])\"$" "\"$1-gzip\""
 </FilesMatch>
</IfModule>

Or, of course, you can just turn off etags entirely:

FileETag None

This is maybe something that IPS should possibly warn people about - I just set up a bog-standard Ubuntu 16.04 server using the default apache build from apt and it still has this bug which just destroys browser caching advantages for anyone reloading a page.

(Also, for the record, the bug wasn't happening with the custom.css because the file was so small that Apache didn't even bother trying to gzip it, so the caching worked correctly for it.)