hc-httpclient-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adam Patacchiola <a...@2fours.com>
Subject Re: 4.2.3 gzip caching still broken?
Date Thu, 11 Apr 2013 19:02:44 GMT
More investigation. The problem seems to arise in BasicHttpCache:

    void storeVariantEntry(
            final HttpHost target,
            final HttpRequest req,
            final HttpCacheEntry entry) throws IOException {
        final String parentURI = uriExtractor.getURI(target, req);
        final String variantURI = uriExtractor.getVariantURI(target, req,
        storage.putEntry(variantURI, entry);

        HttpCacheUpdateCallback callback = new HttpCacheUpdateCallback() {

            public HttpCacheEntry update(HttpCacheEntry existing) throws
IOException {
                return doGetUpdatedParentEntry(
                        req.getRequestLine().getUri(), existing, entry,
                        uriExtractor.getVariantKey(req, entry),


        try {
            storage.updateEntry(parentURI, callback);
        } catch (HttpCacheUpdateException e) {
            log.warn("Could not update key [" + parentURI + "]", e);

the variantURI and parentURI are different. In my storage implementation my
updateEntry looks like:

public void updateEntry(String arg0, HttpCacheUpdateCallback arg1) throws
IOException, HttpCacheUpdateException {
try {
SurespotLog.v(TAG, "updating entry, url: " + arg0);
putEntry(arg0, arg1.update(getEntry(arg0)));
catch (Exception e) {
SurespotLog.w(TAG, "updateEntry", e);


so it ended up adding 2 entries in my cache, one for the parentURI and one
for the variantURI which had the prepended {Accept-Encoding=}.

My updateEntry implementation is probably flawed in that I should get the
entry and check it exists before blindly updating it, but I am curious as
to why BasicHttpCache would put the entry with one key and update the entry
with a different key?



On Thu, Apr 11, 2013 at 12:12 PM, Adam Patacchiola <adam@2fours.com> wrote:

> I have done some more research on this and it appears that the caching is
> working, however it is adding 2 entries to the backing cache: one each with
> and without the url pre-pended by {Accept-Encoding=}. This results in a
> cache miss for the get with the pre-pended url, and uses double the storage
> space in whatever mechanism you are backing the client with. There was a
> bug in my backing store which led to me initially believing it was not
> caching the (correct) url.
> tl;dr it is caching but adding a duplicate invalid entry that never gets
> hit.
> On Thu, Apr 11, 2013 at 10:13 AM, Adam Patacchiola <adam@2fours.com>wrote:
>> I'm using 4.2.3 with gzip compression and CachingHttpClient. Initially I
>> implemented the custom request/response interceptors as described here:
>> https://hc.apache.org/httpcomponents-client-ga/httpclient/examples/org/apache/http/examples/client/ClientGZipContentCompression.java
>> did not work, resulting in the issue described here:
>> https://issues.apache.org/jira/browse/HTTPCLIENT-1163.
>> It appeared to me from reading this issue that using the
>> "CompressionDecorator" would resolve the issue so I modified my code to use
>> DecompressingHttpClient (
>> https://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/impl/client/DecompressingHttpClient.html)
>> but the issue still persists as we can see from the below log output. It is
>> caching using one (broken?) key but then looking it up using a different
>> (correct?) key which is consistent with the bug above:
>> 04-11 09:32:54.760: ... putting cache entry, url: {Accept-Encoding=}
>> https://www.surespot.me:8080/images/b:f1/165
>> 04-11 09:32:55.965: ... Cache miss [host: https://www.surespot.me:8080;
>> uri: https://www.surespot.me:8080/images/b:f1/165]
>> Am I missing something or is this still broken?
>> Thanks,
>> Adam

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message