httpd-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Axel-Stéphane SMORGRAV <Axel-Stephane.SMORG...@europe.adp.com>
Subject [users@httpd] Re: Stuck trying to mod_disk_cache local cgi content
Date Fri, 29 Oct 2004 09:03:57 GMT
Got your dump, and I may have the solution.

It might be a long shot, but worth trying.

I have once used an Apache front-end to a WebLogic application server that used mod_headers
to insert cache-control headers into the responses from WebLogic. If memory serves me right,
I spelled the header "Cache-control". The Apache cache upstream from this front-end did not
recognize the inserted header. 

>From your logs (request and response included at the end of this post), the response to
the request for "GET /dra/DRA_SunSpirals_Element_22A1___1174_224_908_278_1226_782.jpg HTTP/1.1"
contains header names in all lower-caps, including "expires" and "cache-control".

I checked the mod_cache code (2.0.52) to see if it might be case-sensitive:

cache_util.c:    cc_cresp = apr_table_get(r->headers_out, "Cache-Control");
cache_util.c:    cc_ceresp = apr_table_get(r->err_headers_out, "Cache-Control");
cache_util.c:    cc_req = apr_table_get(r->headers_in, "Cache-Control");
cache_util.c:            expstr = apr_table_get(r->headers_out, "Expires");

It does seem like it is case-sensitive, doesn't it?

I also checked RFC2616 withoug beeing able to find anything about case-sensitivity of header
names. Maybe I did not look hard enough, though.

I therefore suggest that :

1. you change your servlet implementation so it serves header with same caps as in the RFC,
i.e. "Cache-Control", "Expires", "Last-Modified" etc... 
2. you read RFC2616 thouroughly to see if it states anything about case-sensitivity for header
names, and if it does not you post a bug report on Bugzilla.

Let us know how that worked out for you!

-ascs


GET /dra/DRA_SunSpirals_Element_22A1___1174_224_908_278_1226_782.jpg HTTP/1.1 
Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/vnd.ms-excel, application/msword,
application/x-shockwave-flash, */* 
Accept-Language: en-us 
Accept-Encoding: gzip, deflate User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1;
SV1) 
Host: javalina:55555 
Connection: Keep-Alive 
Cookie: dra=1094767369; _SID_=20041028190003-dab49249c820654fbd97cb97a50aa258  

HTTP/1.1 200 OK 
Date: Fri, 29 Oct 2004 02:03:29 GMT 
Server: Apache/2.0.51 (Fedora) 
content-length:  11495 
expires:  Fri, 29 Oct 2004 03:03:29 GMT 
last-modified:  Wed, 15 Nov 2003 04:58:08 GMT 
cache-control:  max-age=3600 
content-type:  image/jpeg 
Set-Cookie:  _SID_=20041028190003-dab49249c820654fbd97cb97a50aa258; Path=/; 
Keep-Alive: timeout=15, max=100 
Connection: Keep-Alive

[data snipped]


-----Message d'origine-----
De : Axel-Stéphane SMORGRAV [mailto:Axel-Stephane.SMORGRAV@europe.adp.com] 
Envoyé : jeudi 28 octobre 2004 11:18
À : users@httpd.apache.org
Objet : [users@httpd] RE : [users@httpd] Re: Stuck trying to mod_disk_cache local cgi content

Your dump is not very readable and contains only the beginning of the request and response
sequences. Could you send me a compressed tcpdump that I can analyse with my own tools? I'll
post the conversation on the list. Otherwise Ethereal is really easy to use. You need to do
the capture first, stop the capture and then Tools->Follow TCP Stream. You can use Ethereal
to analyse a capture file from tcpdump or snoop.

What throws me off in your logs is that I do not see anything indicating that mod_cache ever
tries to look for the entity in the cache. Either you have left out a lot of lines, or something
else is not right. I would have expected a message like:

[Thu Oct 28 10:11:49 2004] [debug] mod_cache.c(168): cache: no cache - add cache_in filter
and DECLINE

-ascs

-----Message d'origine-----
De : Roger [mailto:crosseyedpenguin@cox.net] 
Envoyé : jeudi 28 octobre 2004 05:38
À : users@httpd.apache.org
Objet : Re: [users@httpd] Re: Stuck trying to mod_disk_cache local cgi content


Oops, sorry about the 404, I did have one of those along the way and 
obviously copied the wrong log entry. Try 2 below shows a 200. I think I 
have the headers you requested below.

Since I am getting into this much more than I anticipated, I will try 
the same test tomorrow under Fedora Core 2 instead of W/XP to see if 
that makes any difference.

Roger

Here is the next try, access log showing the page with one image on it, 
the one image, a two minute pause while I cleared the browser cache, 
started ethereal, and entered the request for just the image.

192.168.2.10 - - [27/Oct/2004:19:19:01 -0700] "GET /dra/ShowAllRockArt?Klass=3&FolderId=23&Command=Submit&spBB=B1
HTTP/1.1" 
200 5411 "http://192.168.2.11:55555/dra/ShowAllRockArt?Folder=23" "Mozilla/5.0 (Windows; U;
Win 9x 4.90; en-US; rv:1.7) Gecko/20040707 Firefox/0.9.2" 192.168.2.10 - - [27/Oct/2004:19:19:01
-0700] "GET /dra/DRA_SunSpirals_Element_22A1___1174_224_908_278_1226_782_75_0.jpg 
HTTP/1.1" 200 11495 
"http://192.168.2.11:55555/dra/ShowAllRockArt?Klass=3&FolderId=23&Command=Submit&spBB=B1"
"Mozilla/5.0 (Windows; U; Win 9x 4.90; en-US; rv:1.7) Gecko/20040707 Firefox/0.9.2" 192.168.2.10
- - [27/Oct/2004:19:21:33 -0700] "GET /dra/DRA_SunSpirals_Element_22A1___1174_224_908_278_1226_782_75_0.jpg

HTTP/1.1" 200 11495 "-" "Mozilla/5.0 (Windows; U; Win 9x 4.90; en-US; rv:1.7) Gecko/20040707
Firefox/0.9.2"

Next the error log for the above sequence, still shows both images were 
cached rather than the second one being read from the cache.

[Wed Oct 27 19:19:01 2004] [debug] mod_cache.c(506): cache: /dra/ShowAllRockArt?Klass=3&FolderId=23&Command=Submit&spBB=B1
not cached. Reason: Query string present but no expires header [Wed Oct 27 19:19:02 2004]
[info] disk_cache: Storing URL 192.168.2.11/webkit/dra/ShowPix?pid=1174&sz=224&ulx=908&uly=278&lrx=1226&lry=782&iq=75
[Wed Oct 27 19:19:02 2004] [debug] mod_cache.c(613): cache: Caching url: /dra/DRA_SunSpirals_Element_22A1___1174_224_908_278_1226_782_75_0.jpg
[Wed Oct 27 19:19:02 2004] [debug] mod_disk_cache.c(613): disk_cache: Stored headers for URL
192.168.2.11/webkit/dra/ShowPix?pid=1174&sz=224&ulx=908&uly=278&lrx=1226&lry=782&iq=75
[Wed Oct 27 19:19:02 2004] [debug] mod_disk_cache.c(684): disk_cache: Body for URL 192.168.2.11/webkit/dra/ShowPix?pid=1174&sz=224&ulx=908&uly=278&lrx=1226&lry=782&iq=75
cached. [Wed Oct 27 19:21:34 2004] [info] disk_cache: Storing URL 192.168.2.11/webkit/dra/ShowPix?pid=1174&sz=224&ulx=908&uly=278&lrx=1226&lry=782&iq=75
[Wed Oct 27 19:21:34 2004] [debug] mod_cache.c(613): cache: Caching url: /dra/DRA_SunSpirals_Element_22A1___1174_224_908_278_1226_782_75_0.jpg
[Wed Oct 27 19:21:34 2004] [debug] mod_disk_cache.c(613): disk_cache: Stored headers for URL
192.168.2.11/webkit/dra/ShowPix?pid=1174&sz=224&ulx=908&uly=278&lrx=1226&lry=782&iq=75
[Wed Oct 27 19:21:34 2004] [debug] mod_disk_cache.c(684): disk_cache: Body for URL 192.168.2.11/webkit/dra/ShowPix?pid=1174&sz=224&ulx=908&uly=278&lrx=1226&lry=782&iq=75
cached.

Ethereal was a bit intimidating, I kept looking for the large red 
blinking "Follow TCP stream" button on the toolbar but could find none. 
So this is the best I could do by dumping the whole mess to a test file 
and sorting through it.

I think this is the request you are looking for:

Frame 6 (602 bytes on wire, 602 bytes captured)
Ethernet II, Src: 00:0c:41:62:84:8e, Dst: 00:0d:87:a4:6d:0e Internet Protocol, Src Addr: 192.168.2.10
(192.168.2.10), Dst Addr: 
192.168.2.11 (192.168.2.11)
Transmission Control Protocol, Src Port: 1062 (1062), Dst Port: 55555 
(55555), Seq: 1, Ack: 1, Len: 548
Data (548 bytes)

[snip]

--- snipped ----




Axel-Stéphane SMORGRAV wrote:

>What I wanted was for you to request just the JPG through the reverse
>proxy. The reason why I suggested this was to eliminate the possibility 
>for any request headers forcing end-server validation. I see you got a 
>404 return code, so something must have been wrong with the URL... The 
>leading // may be the reason, although I thought it would be 
>canonicalised into a single /.
>
>1. What entities are cacheable is described in RFC2616 section 13.4.
>Basically a document is cached if it contains at least one of 
>Last-Modified, Etag or Expires headers. However, if the document 
>contains a Cache-Control header containing no-store or no-cache (HTTP 
>1.1), or Pragma no-cache (HTTP 1.0) the document should not be cached. 
>A request may contain headers that modify cacheability of an entity. 
>For the gory details read chapter 14.
>
>2. I suggest that a next step should be to somehow grab the HTTP
>exchange on the wire using tcpdump, snoop or Ethereal. Ethereal has a 
>nice feature "Follow TCP stream" that allows you to see the whole 
>conversation including headers, and it can read output from snnop and 
>tcpdump. Then maybe you could post the request and response headers. 
>When you record a conversation, try to keep traffic down by requesting 
>only the JPG and make sure you get a 200 response code. In order for 
>this to work the client (browser) and proxy need to be on separate 
>hosts.
>
>I look forward to seeing those headers.
>Gd lck
>-ascs
>
>-----Message d'origine-----
>De : Roger [mailto:crosseyedpenguin@cox.net]
>Envoyé : mardi 26 octobre 2004 18:52
>À : users@httpd.apache.org
>Objet : Re: [users@httpd] Re: Stuck trying to mod_disk_cache local cgi 
>content
>
>
>I tried accessing the jpg url from my browser. The results are the
>same.
>But I am somewhat confused by your request and I am not sure I did what 
>you expected. The html page containing the image has an image tag of the 
>form:
>    <img 
>src="DRA_SpiralShanty_Element_3A2___1226_224_358_634_806_1240_75_0.jpg" 
>alt=....
>
>The above shows a GET in my access_log:
>127.0.0.1 - - [26/Oct/2004:08:56:16 -0700] "GET 
>/dra/DRA_SpiralShanty_Element_3A2___1226_224_358_634_806_1240_75_0.jpg
>HTTP/1.1" 200 11612
>"http://foxdigitalrockart.org:55555/dra/ShowAllRockArt?Klass=14&FolderId=26&Command=Submit&spBB=B7"

>"Mozilla/5.0 (Windows; U; Windows NT 5.1; rv:1.7.3) Gecko/20040913 
>Firefox/0.10"
>
>When I put the jpg url into my browser directly, the access_log shows a 
>similar GET entry, less the referencing page: 127.0.0.1 - - 
>[26/Oct/2004:09:04:54 -0700] "GET 
>//dra/DRA_SpiralShanty_Element_3A2___1226_224_358_634_806_1240_75_0.jpg
>HTTP/1.1" 404 274 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1;
>rv:1.7.3) Gecko/20040913 Firefox/0.10"
>
>Putting in the RewriteRule (described earlier) was the way I managed to 
>get browsers to default to a nicer (not perfect) file name should the 
>caller do a right-click and choose save as.
>
>I also have instances where I do not make use of the RewriteRule. The 
>html pages have img tages like:
>    <img src="ShowPix?pid=842&amp;tn=192" alt="">
>
>The results from the above are the same. The images are rewritten to
>the
>cache on every request but never read from the cache. All the browsers I 
>test with retain the images in their cache and do not request a fresh 
>copy unless I clear the cache.
>
>Is there any doc that details what headers are required to be present 
>for mod_cache and is there an easy way see the incoming headers for an 
>image request to determine that I have them all set correctly?
>
>Roger
>
>
>Axel-Stéphane SMORGRAV wrote:
>
>  
>
>>What happens when you do just a "GET 
>>/dra/DRA_SpiralShanty_Element_3A2___1226_224_358_634_806_1240_75_0.jpg
>>"
>>i.e. request URL 
>>http://a.b.c/dra/DRA_SpiralShanty_Element_3A2___1226_224_358_634_806_12
>>40_75_0.jpg? Is it then served from the cache?
>>
>>-ascs
>>
>>-----Message d'origine-----
>>De : Roger [mailto:crosseyedpenguin@cox.net]
>>Envoyé : lundi 25 octobre 2004 19:34
>>À : users@httpd.apache.org
>>Objet : Re: [users@httpd] RE : [users@httpd] Stuck trying to
>>mod_disk_cache local cgi content
>>
>>
>>Axel-Stéphane SMORGRAV wrote:
>>
>> 
>>
>>    
>>
>>>Could you put your virtual host into LogLevel debug and post the
>>>relevant parts of the error_log??
>>>
>>>-ascs
>>>
>>>-----Message d'origine-----
>>>De : Roger [mailto:crosseyedpenguin@cox.net]
>>>Envoyé : dimanche 17 octobre 2004 21:24
>>>À : users@httpd.apache.org
>>>Objet : [users@httpd] Stuck trying to mod_disk_cache local cgi
>>>content
>>>
>>>
>>>Hi,
>>>
>>>I have a python cgi application which modifies .jpg files on the fly. 
>>>I am trying to implement disk caching for these images. I am running 
>>>Windows/XP with Apache 2.0.52.
>>>
>>>The html pages created refer to the images as if they were .jpg
>>>images
>>>and I use a rewrite rule to transform the request into a cgi get 
>>>request. So my first change was to just change the rewrite rule to 
>>>force a proxy request:
>>>  RewriteRule (^/dra/DRA_.*)___(.*)_(.*)(\.jpg$)
>>>http://mysite.org/webkit/dra/ShowPix\?pid=$2\&tn=$3 [P,L]
>>>
>>>(The original purpose of the above rule was to provide a pretty 
>>>default file name should the user right-click and do a save as.)
>>>
>>>Per the docs, I loaded the modules and created the proxy and disk 
>>>caching statements:
>>>  LoadModule proxy_module modules/mod_proxy.so
>>>  LoadModule proxy_http_module modules/mod_proxy_http.so
>>>  LoadModule cache_module modules/mod_cache.so
>>>
>>>  ProxyRequests Off
>>>  <ProxyMatch .*>
>>>      Order deny,allow
>>>      Allow from all
>>>  </ProxyMatch>
>>>  LoadModule disk_cache_module modules/mod_disk_cache.so
>>>      CacheRoot c:/ApacheCache
>>>      CacheSize 1000000
>>>      CacheEnable disk /
>>>      CacheDirLevels 5
>>>      CacheDirLength 3
>>>
>>>The result of the above is that Apache is caching some local .css,
>>>and
>>>.js files but is not caching the .jpg files covered by the rewrite 
>>>rule. I plan on turning off the .css and .js caching with the 
>>>proxymatch directive later.
>>>
>>>The script creating the .jpg images puts out headers for
>>>('cache-control','max-age=3600') and the browsers I am testing with 
>>>(IE, Mozilla, Opera) all cache the images successfully. So everything 
>>>appears to be working OK except the .jpg files are not cached by 
>>>Apache.
>>>
>>>Anyone have a clue as to what I am doing wrong?
>>>
>>>Roger
>>>
>>>
>>>
>>>
>>>   
>>>
>>>      
>>>
>>Thank you, I am closer to the solution. The files are now being
>>cached, but not served from the cache.
>>
>>After adding LogLevel  debug to the virtual host, the first message I
>>encountered was:
>>  ...not cached. Reason: Query string present but no expires header
>>
>>So I added an expires header: equal to now plus 2 days: ('Expires',
>>'Wed, 27 Oct 2004 17:43:29 GMT')
>>
>>So now my images are being cached per the log, but when I view a
>>particular page with these images, delete all tempory internet files, 
>>and view the same page again, the files are not served from the cache.
>>
>>The log with my comments is below.
>>
>>The rewrite rule to change requests for a .jpg file into a querey is
>>as
>>follows:
>>   RewriteRule 
>>(^/dra/DRA_.*)___(.*)_(.*)_(.*)_(.*)_(.*)_(.*)_(.*)_(.*)(\.jpg$)
>>        
>>/webkit/dra/ShowPix\?pid=$2\&sz=$3\&ulx=$4\&uly=$5\&lrx=$6\&lry=$7\&iq
>>=
>>$8
>>[L,PT]
>>
>>Roger
>>
>>#
>>#  First access, nothing cached, this is the request for a page
>>containing a .jpg image #
>>[Mon Oct 25 10:14:05 2004] [debug] mod_cache.c(506): cache: 
>>/dra/ShowAllRockArt?Klass=14&FolderId=26&Command=Submit&spBB=B6 not 
>>cached. Reason: Query string present but no expires header
>>#
>># This is activity for the target jpg image, looks as if it is getting 
>>cached properly
>>#
>>[Mon Oct 25 10:14:05 2004] [info] disk_cache: Storing URL 
>>foxdigitalrockart.org/webkit/dra/ShowPix?pid=1226&sz=224&ulx=358&uly=634&lrx=806&lry=1240&iq=75
>>[Mon Oct 25 10:14:05 2004] [debug] mod_cache.c(613): cache: Caching url: 
>>/dra/DRA_SpiralShanty_Element_3A2___1226_224_358_634_806_1240_75_0.jpg
>>[Mon Oct 25 10:14:05 2004] [debug] mod_disk_cache.c(613): disk_cache: 
>>Stored headers for URL 
>>foxdigitalrockart.org/webkit/dra/ShowPix?pid=1226&sz=224&ulx=358&uly=634&lrx=806&lry=1240&iq=75
>>[Mon Oct 25 10:14:06 2004] [debug] mod_disk_cache.c(684): disk_cache: 
>>Body for URL 
>>foxdigitalrockart.org/webkit/dra/ShowPix?pid=1226&sz=224&ulx=358&uly=634&lrx=806&lry=1240&iq=75

>>cached.
>>#
>># browser cache cleared, second try for same page, this is extra junk 
>>getting into position for request
>>#
>>[Mon Oct 25 10:14:22 2004] [debug] mod_cache.c(506): cache: 
>>/dra/ShowAllRockArt?Command=Change+Class&spBB=B7 not cached. Reason: 
>>Query string present but no expires header
>>[Mon Oct 25 10:14:22 2004] [debug] mod_disk_cache.c(400): disk_cache: 
>>Recalled cached URL info header foxdigitalrockart.org/drastatic/ra1.css?
>>[Mon Oct 25 10:14:22 2004] [debug] mod_disk_cache.c(400): disk_cache: 
>>Recalled cached URL info header foxdigitalrockart.org/drastatic/buttons.js?
>>[Mon Oct 25 10:14:22 2004] [debug] mod_disk_cache.c(464): disk_cache: 
>>Recalled headers for URL foxdigitalrockart.org/drastatic/ra1.css?
>>[Mon Oct 25 10:14:22 2004] [debug] mod_cache.c(285): cache: running 
>>CACHE_CONDITIONAL filter
>>[Mon Oct 25 10:14:22 2004] [debug] mod_disk_cache.c(464): disk_cache: 
>>Recalled headers for URL foxdigitalrockart.org/drastatic/buttons.js?
>>[Mon Oct 25 10:14:22 2004] [debug] mod_cache.c(285): cache: running 
>>CACHE_CONDITIONAL filter
>>[Mon Oct 25 10:14:22 2004] [debug] mod_disk_cache.c(400): disk_cache: 
>>Recalled cached URL info header foxdigitalrockart.org/drastatic/logosml.jpg?
>>[Mon Oct 25 10:14:22 2004] [debug] mod_disk_cache.c(400): disk_cache: 
>>Recalled cached URL info header 
>>foxdigitalrockart.org/drastatic/wz_tooltip.js?
>>[Mon Oct 25 10:14:22 2004] [debug] mod_disk_cache.c(464): disk_cache: 
>>Recalled headers for URL foxdigitalrockart.org/drastatic/logosml.jpg?
>>[Mon Oct 25 10:14:22 2004] [debug] mod_cache.c(285): cache: running 
>>CACHE_CONDITIONAL filter
>>[Mon Oct 25 10:14:22 2004] [debug] mod_disk_cache.c(464): disk_cache: 
>>Recalled headers for URL foxdigitalrockart.org/drastatic/wz_tooltip.js?
>>[Mon Oct 25 10:14:22 2004] [debug] mod_cache.c(285): cache: running 
>>CACHE_CONDITIONAL filter
>>#
>># Here comes request for same page
>>#
>>[Mon Oct 25 10:14:26 2004] [debug] mod_cache.c(506): cache: 
>>/dra/ShowAllRockArt?Klass=14&FolderId=26&Command=Submit&spBB=B8 not 
>>cached. Reason: Query string present but no expires header
>>#
>># This image should have been served from cache, but instead it was 
>>recreated and stored again
>>#
>>[Mon Oct 25 10:14:27 2004] [info] disk_cache: Storing URL 
>>foxdigitalrockart.org/webkit/dra/ShowPix?pid=1226&sz=224&ulx=358&uly=634&lrx=806&lry=1240&iq=75
>>[Mon Oct 25 10:14:27 2004] [debug] mod_cache.c(613): cache: Caching url: 
>>/dra/DRA_SpiralShanty_Element_3A2___1226_224_358_634_806_1240_75_0.jpg
>>[Mon Oct 25 10:14:27 2004] [debug] mod_disk_cache.c(613): disk_cache: 
>>Stored headers for URL 
>>foxdigitalrockart.org/webkit/dra/ShowPix?pid=1226&sz=224&ulx=358&uly=634&lrx=806&lry=1240&iq=75
>>[Mon Oct 25 10:14:27 2004] [debug] mod_disk_cache.c(684): disk_cache: 
>>Body for URL 
>>foxdigitalrockart.org/webkit/dra/ShowPix?pid=1226&sz=224&ulx=358&uly=634&lrx=806&lry=1240&iq=75

>>cached.
>>
>>    
>>
>
>  
>



---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project. See <URL:http://httpd.apache.org/userslist.html>
for more info. To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
   "   from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org


---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project. See <URL:http://httpd.apache.org/userslist.html>
for more info. To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
   "   from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org



---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
   "   from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org


Mime
View raw message