Cache sharing between sites

re: Cache sharing between sites

Bertrand Le Roy — Wed, 28 May 2008 23:29:43 GMT

@Jonah: Some have expressed privacy concerns over having scripts hosted by Google (which then gets a lot of free information about people browsing your site through the referrer header). You also have to trust the central location to always be available. But yes, centralized hosting by Google and others is a step in the right direction.

I would prefer a solution such as the ones described here because there is no such reliance on a centralized location, yet the scripts get cached across sites and the load is naturally distributed.

re: Cache sharing between sites

Jonah Dempcy — Thu, 08 May 2008 02:49:12 GMT

Why not just rely on Google to provide the bandwidth and simply have them host all the major JS frameworks? They are already hosting plenty of code snippets on Google Code and they offer hotlinks (the link actually has the text "Hotlink/Download" so they intend for you to use it directly if you so desire).

For example, I have used Dean Edwards' IE7.js library. Normally I would concatenate it together with all the other JS as part of the build process to minimize I/O traffic, but I'm wondering if I'd be better off just hotlinking to Google Code, on the off-chance that other sites are doing the same and the download would be cached:

ie7-js.googlecode.com/.../2.0(beta)/IE7.js

For now, I will keep doing my concatenation thing because minimizing the amount of separate JS files seems to have the greatest benefit (and I doubt many people have the IE7.js file cached). But, if Google Code were to host MooTools, jQuery or Prototype I'd be all over it in a flash. I'd stop having to write build scripts that combine those files with the rest of the site JS and be able to just hotlink from Google, basking in the benefits of universal caching for popular pieces of code.

re: Cache sharing between sites

Kurisu — Wed, 09 Apr 2008 23:13:29 GMT

I hadn't actually read the linked articles before commenting. Now I see he even mentions base32 encoding of the hash which kinda shows where Doug got this idea from.

I think what Brendan meant is similar to DTD URIs in XML. These can be normal HTTP URLs and it's certainly useful to avoid namespace clashes. A few parsers do/did really try to fetch the DTD from such URLs if it wasn't stored locally. This can easily cause an unintended DDoS. Likewise it's only safe if you can trust DNS and the server. So signing the scripts using PKI and/or use of TLS is probably implicitly given. Anything would mean someone misunderstood cross-site scripting. If you use only signatures i.e., you trust everything signed with certain keys but no explicit hashes that means of course the script can be modified by the key owner for good or for bad.

The DDoS issue could be circumvented by using Coral: www.coralcdn.org

All in all, I believe both suggestions are actually complementary.

This article is also related and somewhat interesting:

changelog.ca/.../gnutella_does_not_need_the_x-alt_http_header

re: Cache sharing between sites

Bertrand Le Roy — Tue, 08 Apr 2008 21:14:49 GMT

@Kurisu: thanks for the pointers. I agree that crypto hash attacks are blown way out of proportion by Brendan, who tends to generate a quite powerful RDF. That he would be using a security argument to dismiss Doug's approach and propose his own approach which has a much bigger security problem is quite puzzling. And yes, Doug's post wasn't limited to script and neither should any implementation of that stuff. CSS and images would benefit from the same optimizations. To cite the post above, "all elements that have a "src" or "href" attribute".

re: Cache sharing between sites

Kurisu — Tue, 08 Apr 2008 16:55:46 GMT

A lot of storage systems use exactly this concept i.e., storing hashes of files or chunks thereof along with the files themselves and use this to avoid transferring or sometimes even storing the same data more than once. Not surprising very similar ideas have been around for years and decades. Consider the ETag header, the Content-MD5 header or much closer RFC 2169:

www.faqs.org/.../rfc2169.html

The Gnutella network takes advantage of this RFC.

Calling cryptographic hashes too weak for this purpose borders on reality distortion field. How do these people think cryptographic signatures or SSL/TLS works? Not to mention that attacking the hashes by trying to generate evil twins is the least probable attack vector. Anyway, in this context though I'd consider it a total pain for maintenence. At the very least it would require dedicated tools to keep everything consistent during updates. Also if I was going through all this hassle I'd extend this to all files not just JavaScript. For browsers or proxies that already implement caching this would be a low-hanging fruit anyway. All they need to add is an index with the hashes for the cached files.

re: Cache sharing between sites

Bertrand Le Roy — Sat, 05 Apr 2008 21:52:17 GMT

@Joe: yes, I actually read Brendan's answer before I wrote that post, but I don't see how it answers my question. I reformulated with another comment that for some reason he chose not to publish. That's why I posted my comments on my own blog ;) The attack scenario that I describe in this post, unless I'm mistaken, is not mitigated by his answer in any way as the src value would never be hit if something is already in the cache for the shared value, which may be a malicious script.

@Philip: I think the third solution that I describe doesn't have this flaw at all. But Doug's approach is much more difficult to compromise than Brendan's: the scenario with his approach is that the cache is organized as a dictionary of hashes to scripts. When you visit a page, if there is a script with a hash attribute, the browser looks up that hash and if it finds it it uses the script. But the way the cache is being constructed is what guarantees the integrity of the hash: the first time a script is loaded, the browser computes the hash locally from the contents and uses the computed hash as the key in the dictionary. Does that clarify?

re: Cache sharing between sites

Philip — Sat, 05 Apr 2008 18:51:45 GMT

I don't see the security difference between the two plans: how do you guarantee that the hash "generated with a well-defined cryptographic hash algorithm" really matches the javascript file referenced by the src attribute? To do that, you would have to have the browser compute the hash after receiving the javascript file (and then, why are you specifying it in the html in the first place?).

The two plans are equally insecure and share the attack that you describe.

re: Cache sharing between sites

Joe Chung — Sat, 05 Apr 2008 05:18:17 GMT

Brendan responded to your question in his blog. For your readers' convenience:

"Only the @shared value would be shared among script tags. The @src would be loaded only if there was no cache entry for @shared."