hc-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From j...@apache.org
Subject svn commit: r1376181 - in /httpcomponents/httpclient/trunk: ./ httpclient-cache/src/main/java/org/apache/http/client/cache/ httpclient-cache/src/main/java/org/apache/http/impl/client/cache/ httpclient-cache/src/test/java/org/apache/http/impl/client/cac...
Date Wed, 22 Aug 2012 18:31:53 GMT
Author: jonm
Date: Wed Aug 22 18:31:52 2012
New Revision: 1376181

URL: http://svn.apache.org/viewvc?rev=1376181&view=rev
Log:
HTTPCLIENT-1223: Forward port from 4.2.X branch to trunk.

Modified:
    httpcomponents/httpclient/trunk/   (props changed)
    httpcomponents/httpclient/trunk/RELEASE_NOTES.txt
    httpcomponents/httpclient/trunk/httpclient-cache/src/main/java/org/apache/http/client/cache/HttpCacheStorage.java
  (props changed)
    httpcomponents/httpclient/trunk/httpclient-cache/src/main/java/org/apache/http/impl/client/cache/BasicHttpCacheStorage.java
  (props changed)
    httpcomponents/httpclient/trunk/httpclient-cache/src/main/java/org/apache/http/impl/client/cache/CacheInvalidator.java
    httpcomponents/httpclient/trunk/httpclient-cache/src/main/java/org/apache/http/impl/client/cache/CachingHttpClient.java
    httpcomponents/httpclient/trunk/httpclient-cache/src/test/java/org/apache/http/impl/client/cache/TestCacheInvalidator.java
    httpcomponents/httpclient/trunk/httpclient-cache/src/test/java/org/apache/http/impl/client/cache/TestProtocolRecommendations.java
    httpcomponents/httpclient/trunk/httpclient-contrib/src/main/java/org/apache/http/contrib/auth/BouncySpnegoTokenGenerator.java
  (props changed)
    httpcomponents/httpclient/trunk/httpclient/src/test/resources/test.keystore   (props changed)
    httpcomponents/httpclient/trunk/src/docbkx/caching.xml
    httpcomponents/httpclient/trunk/src/docbkx/resources/   (props changed)
    httpcomponents/httpclient/trunk/src/site/apt/ntlm.apt   (props changed)

Propchange: httpcomponents/httpclient/trunk/
------------------------------------------------------------------------------
  Merged /httpcomponents/httpclient/branches/4.2.x:r1376150-1376151

Modified: httpcomponents/httpclient/trunk/RELEASE_NOTES.txt
URL: http://svn.apache.org/viewvc/httpcomponents/httpclient/trunk/RELEASE_NOTES.txt?rev=1376181&r1=1376180&r2=1376181&view=diff
==============================================================================
--- httpcomponents/httpclient/trunk/RELEASE_NOTES.txt (original)
+++ httpcomponents/httpclient/trunk/RELEASE_NOTES.txt Wed Aug 22 18:31:52 2012
@@ -1,10 +1,12 @@
 Changes since 4.2.1 
 -------------------
 
+* [HTTPCLIENT-1223] Cache could be more aggressive on cache invalidations
+  from Content-Location. Contributed by Jon Moore <jonm at apache.org>.
+
 * [HTTPCLIENT-1216] Added method to force clean thread-local used by DateUtils.
   Contributed by Oleg Kalnichevski <olegk at apache.org>
 
-
 Release 4.2.1 
 -------------------
 

Propchange: httpcomponents/httpclient/trunk/httpclient-cache/src/main/java/org/apache/http/client/cache/HttpCacheStorage.java
------------------------------------------------------------------------------
  Merged /httpcomponents/httpclient/branches/4.2.x/httpclient-cache/src/main/java/org/apache/http/client/cache/HttpCacheStorage.java:r1376150-1376151

Propchange: httpcomponents/httpclient/trunk/httpclient-cache/src/main/java/org/apache/http/impl/client/cache/BasicHttpCacheStorage.java
------------------------------------------------------------------------------
  Merged /httpcomponents/httpclient/branches/4.2.x/httpclient-cache/src/main/java/org/apache/http/impl/client/cache/BasicHttpCacheStorage.java:r1376150-1376151

Modified: httpcomponents/httpclient/trunk/httpclient-cache/src/main/java/org/apache/http/impl/client/cache/CacheInvalidator.java
URL: http://svn.apache.org/viewvc/httpcomponents/httpclient/trunk/httpclient-cache/src/main/java/org/apache/http/impl/client/cache/CacheInvalidator.java?rev=1376181&r1=1376180&r2=1376181&view=diff
==============================================================================
--- httpcomponents/httpclient/trunk/httpclient-cache/src/main/java/org/apache/http/impl/client/cache/CacheInvalidator.java
(original)
+++ httpcomponents/httpclient/trunk/httpclient-cache/src/main/java/org/apache/http/impl/client/cache/CacheInvalidator.java
Wed Aug 22 18:31:52 2012
@@ -199,7 +199,10 @@ class CacheInvalidator {
         HttpCacheEntry entry = getEntry(cacheKey);
         if (entry == null) return;
 
-        if (!responseDateNewerThanEntryDate(response, entry)) return;
+        // do not invalidate if response is strictly older than entry
+        // or if the etags match
+        
+        if (responseDateOlderThanEntryDate(response, entry)) return;
         if (!responseAndEntryEtagsDiffer(response, entry)) return;
 
         flushUriIfSameHost(reqURL, canonURL);
@@ -222,18 +225,20 @@ class CacheInvalidator {
         return (!entryEtag.getValue().equals(responseEtag.getValue()));
     }
 
-    private boolean responseDateNewerThanEntryDate(HttpResponse response,
+    private boolean responseDateOlderThanEntryDate(HttpResponse response,
             HttpCacheEntry entry) {
         Header entryDateHeader = entry.getFirstHeader(HTTP.DATE_HEADER);
         Header responseDateHeader = response.getFirstHeader(HTTP.DATE_HEADER);
         if (entryDateHeader == null || responseDateHeader == null) {
+            /* be conservative; should probably flush */
             return false;
         }
         try {
             Date entryDate = DateUtils.parseDate(entryDateHeader.getValue());
             Date responseDate = DateUtils.parseDate(responseDateHeader.getValue());
-            return responseDate.after(entryDate);
+            return responseDate.before(entryDate);
         } catch (DateParseException e) {
+            /* flushing is always safe */
             return false;
         }
     }

Modified: httpcomponents/httpclient/trunk/httpclient-cache/src/main/java/org/apache/http/impl/client/cache/CachingHttpClient.java
URL: http://svn.apache.org/viewvc/httpcomponents/httpclient/trunk/httpclient-cache/src/main/java/org/apache/http/impl/client/cache/CachingHttpClient.java?rev=1376181&r1=1376180&r2=1376181&view=diff
==============================================================================
--- httpcomponents/httpclient/trunk/httpclient-cache/src/main/java/org/apache/http/impl/client/cache/CachingHttpClient.java
(original)
+++ httpcomponents/httpclient/trunk/httpclient-cache/src/main/java/org/apache/http/impl/client/cache/CachingHttpClient.java
Wed Aug 22 18:31:52 2012
@@ -426,11 +426,13 @@ public class CachingHttpClient implement
         flushEntriesInvalidatedByRequest(target, request);
 
         if (!cacheableRequestPolicy.isServableFromCache(request)) {
+            log.debug("Request is not servable from cache");
             return callBackend(target, request, context);
         }
 
         HttpCacheEntry entry = satisfyFromCache(target, request);
         if (entry == null) {
+            log.debug("Cache miss");
             return handleCacheMiss(target, request, context);
         }
 
@@ -444,12 +446,16 @@ public class CachingHttpClient implement
         HttpResponse out = null;
         Date now = getCurrentDate();
         if (suitabilityChecker.canCachedResponseBeUsed(target, request, entry, now)) {
+            log.debug("Cache hit");
             out = generateCachedResponse(request, context, entry, now);
         } else if (!mayCallBackend(request)) {
+            log.debug("Cache entry not suitable but only-if-cached requested");
             out = generateGatewayTimeout(context);
         } else if (validityPolicy.isRevalidatable(entry)) {
+            log.debug("Revalidating cache entry");
             return revalidateCacheEntry(target, request, context, entry, now);
         } else {
+            log.debug("Cache entry not usable; calling backend");
         	return callBackend(target, request, context);
         }
         if (context != null) {
@@ -464,12 +470,12 @@ public class CachingHttpClient implement
     private HttpResponse revalidateCacheEntry(HttpHost target,
             HttpRequest request, HttpContext context, HttpCacheEntry entry,
             Date now) throws ClientProtocolException {
-        log.trace("Revalidating the cache entry");
 
         try {
             if (asynchRevalidator != null
                 && !staleResponseNotAllowed(request, entry, now)
                 && validityPolicy.mayReturnStaleWhileRevalidating(entry, now)) {
+                log.trace("Serving stale with asynchronous revalidation");
                 final HttpResponse resp = generateCachedResponse(request, context, entry,
now);
 
                 asynchRevalidator.revalidateCacheEntry(target, request, context, entry);
@@ -615,6 +621,7 @@ public class CachingHttpClient implement
         for (Header h: request.getHeaders(HeaderConstants.CACHE_CONTROL)) {
             for (HeaderElement elt : h.getElements()) {
                 if ("only-if-cached".equals(elt.getName())) {
+                    log.trace("Request marked only-if-cached");
                     return false;
                 }
             }

Modified: httpcomponents/httpclient/trunk/httpclient-cache/src/test/java/org/apache/http/impl/client/cache/TestCacheInvalidator.java
URL: http://svn.apache.org/viewvc/httpcomponents/httpclient/trunk/httpclient-cache/src/test/java/org/apache/http/impl/client/cache/TestCacheInvalidator.java?rev=1376181&r1=1376180&r2=1376181&view=diff
==============================================================================
--- httpcomponents/httpclient/trunk/httpclient-cache/src/test/java/org/apache/http/impl/client/cache/TestCacheInvalidator.java
(original)
+++ httpcomponents/httpclient/trunk/httpclient-cache/src/test/java/org/apache/http/impl/client/cache/TestCacheInvalidator.java
Wed Aug 22 18:31:52 2012
@@ -401,10 +401,10 @@ public class TestCacheInvalidator {
     }
 
     @Test
-    public void doesNotFlushEntrySpecifiedByContentLocationIfNotNewer()
+    public void doesNotFlushEntrySpecifiedByContentLocationIfOlder()
             throws Exception {
         response.setHeader("ETag","\"new-etag\"");
-        response.setHeader("Date", formatDate(now));
+        response.setHeader("Date", formatDate(tenSecondsAgo));
         String theURI = "http://foo.example.com:80/bar";
         response.setHeader("Content-Location", theURI);
 
@@ -475,7 +475,7 @@ public class TestCacheInvalidator {
     }
 
     @Test
-    public void doesNotFlushEntrySpecifiedByContentLocationIfResponseHasNoDate()
+    public void flushesEntrySpecifiedByContentLocationIfResponseHasNoDate()
             throws Exception {
         response.setHeader("ETag", "\"new-etag\"");
         response.removeHeaders("Date");
@@ -488,6 +488,7 @@ public class TestCacheInvalidator {
         });
 
         expect(mockStorage.getEntry(theURI)).andReturn(entry).anyTimes();
+        mockStorage.removeEntry(theURI);
 
         replayMocks();
         impl.flushInvalidatedCacheEntries(host, request, response);
@@ -495,7 +496,7 @@ public class TestCacheInvalidator {
     }
 
     @Test
-    public void doesNotFlushEntrySpecifiedByContentLocationIfEntryHasNoDate()
+    public void flushesEntrySpecifiedByContentLocationIfEntryHasNoDate()
             throws Exception {
         response.setHeader("ETag","\"new-etag\"");
         response.setHeader("Date", formatDate(now));
@@ -507,6 +508,7 @@ public class TestCacheInvalidator {
         });
 
         expect(mockStorage.getEntry(theURI)).andReturn(entry).anyTimes();
+        mockStorage.removeEntry(theURI);
 
         replayMocks();
         impl.flushInvalidatedCacheEntries(host, request, response);
@@ -514,7 +516,7 @@ public class TestCacheInvalidator {
     }
 
     @Test
-    public void doesNotFlushEntrySpecifiedByContentLocationIfResponseHasMalformedDate()
+    public void flushesEntrySpecifiedByContentLocationIfResponseHasMalformedDate()
             throws Exception {
         response.setHeader("ETag","\"new-etag\"");
         response.setHeader("Date", "blarg");
@@ -527,6 +529,7 @@ public class TestCacheInvalidator {
         });
 
         expect(mockStorage.getEntry(theURI)).andReturn(entry).anyTimes();
+        mockStorage.removeEntry(theURI);
 
         replayMocks();
         impl.flushInvalidatedCacheEntries(host, request, response);
@@ -534,7 +537,7 @@ public class TestCacheInvalidator {
     }
 
     @Test
-    public void doesNotFlushEntrySpecifiedByContentLocationIfEntryHasMalformedDate()
+    public void flushesEntrySpecifiedByContentLocationIfEntryHasMalformedDate()
             throws Exception {
         response.setHeader("ETag","\"new-etag\"");
         response.setHeader("Date", formatDate(now));
@@ -547,6 +550,7 @@ public class TestCacheInvalidator {
         });
 
         expect(mockStorage.getEntry(theURI)).andReturn(entry).anyTimes();
+        mockStorage.removeEntry(theURI);
 
         replayMocks();
         impl.flushInvalidatedCacheEntries(host, request, response);

Modified: httpcomponents/httpclient/trunk/httpclient-cache/src/test/java/org/apache/http/impl/client/cache/TestProtocolRecommendations.java
URL: http://svn.apache.org/viewvc/httpcomponents/httpclient/trunk/httpclient-cache/src/test/java/org/apache/http/impl/client/cache/TestProtocolRecommendations.java?rev=1376181&r1=1376180&r2=1376181&view=diff
==============================================================================
--- httpcomponents/httpclient/trunk/httpclient-cache/src/test/java/org/apache/http/impl/client/cache/TestProtocolRecommendations.java
(original)
+++ httpcomponents/httpclient/trunk/httpclient-cache/src/test/java/org/apache/http/impl/client/cache/TestProtocolRecommendations.java
Wed Aug 22 18:31:52 2012
@@ -1752,5 +1752,27 @@ public class TestProtocolRecommendations
 
         assertTrue(HttpTestUtils.semanticallyTransparent(resp1, result));
     }
+    
+    @Test
+    public void issues304EvenWithWeakETag() throws Exception {
+        HttpRequest req1 = HttpTestUtils.makeDefaultRequest();
+        HttpResponse resp1 = HttpTestUtils.make200Response();
+        resp1.setHeader("Date", formatDate(tenSecondsAgo));
+        resp1.setHeader("Cache-Control", "max-age=300");
+        resp1.setHeader("ETag","W/\"weak-sauce\"");
+        
+        backendExpectsAnyRequest().andReturn(resp1);
+        
+        HttpRequest req2 = HttpTestUtils.makeDefaultRequest();
+        req2.setHeader("If-None-Match","W/\"weak-sauce\"");
+        
+        replayMocks();
+        impl.execute(host, req1);
+        HttpResponse result = impl.execute(host, req2);
+        verifyMocks();
+        
+        assertEquals(HttpStatus.SC_NOT_MODIFIED, result.getStatusLine().getStatusCode());
+                       
+    }
 
 }

Propchange: httpcomponents/httpclient/trunk/httpclient-contrib/src/main/java/org/apache/http/contrib/auth/BouncySpnegoTokenGenerator.java
------------------------------------------------------------------------------
  Merged /httpcomponents/httpclient/branches/4.2.x/httpclient-contrib/src/main/java/org/apache/http/contrib/auth/BouncySpnegoTokenGenerator.java:r1376150-1376151

Propchange: httpcomponents/httpclient/trunk/httpclient/src/test/resources/test.keystore
------------------------------------------------------------------------------
  Merged /httpcomponents/httpclient/branches/4.2.x/httpclient/src/test/resources/test.keystore:r1376150-1376151

Modified: httpcomponents/httpclient/trunk/src/docbkx/caching.xml
URL: http://svn.apache.org/viewvc/httpcomponents/httpclient/trunk/src/docbkx/caching.xml?rev=1376181&r1=1376180&r2=1376181&view=diff
==============================================================================
--- httpcomponents/httpclient/trunk/src/docbkx/caching.xml (original)
+++ httpcomponents/httpclient/trunk/src/docbkx/caching.xml Wed Aug 22 18:31:52 2012
@@ -123,17 +123,13 @@
   <section id="rfc2616compliance">
     <title>RFC-2616 Compliance</title>
 
-    <para>HttpClient Cache makes an effort to be at least <emphasis>conditionally
+    <para>We believe HttpClient Cache is <emphasis>unconditionally
     compliant</emphasis> with <ulink
     url="http://www.ietf.org/rfc/rfc2616.txt">RFC-2616</ulink>. That is,
-    wherever the specification indicates MUST or MUST NOT for HTTP caches, the
-    caching layer attempts to behave in a way that satisfies those
-    requirements. This means the caching module won't produce incorrect
-    behavior when you drop it in. At the same time, the project is continuing
-    to work on unconditional compliance, which would add compliance with all the
-    SHOULDs and SHOULD NOTs, many of which we already comply with. We just can't
-    claim fully unconditional compliance until we satisfy <emphasis>all</emphasis>
-    of them.</para>
+    wherever the specification indicates MUST, MUST NOT, SHOULD, or SHOULD NOT
+    for HTTP caches, the caching layer attempts to behave in a way that satisfies
+    those requirements. This means the caching module won't produce incorrect
+    behavior when you drop it in. </para>
   </section>
 
   <section>
@@ -226,7 +222,7 @@ case VALIDATED:
     offers high performance, it may not be appropriate for your application due to
     the limitation on size or because the cache entries are ephemeral and don't
     survive an application restart. The current release includes support for storing
-    cache entries using Ehcache and memcached implementations, which allow for
+    cache entries using EhCache and memcached implementations, which allow for
     spilling cache entries to disk or storing them in an external process.</para>
     
     <para>If none of those options are suitable for your application, it is

Propchange: httpcomponents/httpclient/trunk/src/docbkx/resources/
------------------------------------------------------------------------------
  Merged /httpcomponents/httpclient/branches/4.2.x/src/docbkx/resources:r1376150-1376151

Propchange: httpcomponents/httpclient/trunk/src/site/apt/ntlm.apt
------------------------------------------------------------------------------
  Merged /httpcomponents/httpclient/branches/4.2.x/src/site/apt/ntlm.apt:r1376150-1376151



Mime
View raw message