httpd-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Abu Hurayrah <>
Subject [users@httpd] MSN-Bot doesn't finish a download?
Date Mon, 02 May 2005 23:27:30 GMT
Greets to all!

I've just noticed that when the MSNbot crawls my website and hits some 
of my downloads, it doesn't download the whole file.

The request is HTTP 1.0, rather than the usual 1.1, and the amount 
downloaded changes depending on a certain factor.

I manage downloads via my own PHP script, which reads in a chunk of a 
file, sends that chunk to the browser, and then continues in a loop 
until the entire file has been sent.

MSN seems to only catch ONE chunk, no matter what size I make it, which 
I find very strange, because I cannot think of why my implementation 
would matter to MSN or not.

Previous to this, MSN used to be one of the chief eaters of my 
bandwidth, ironically, when I used another technique for delivering files.

Can anyone explain this odd behavior, and/or why the HTTP 1.0 GET 
protocal doesn't finish the download on this particular implementation?

Thanks in advance!

View raw message