httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brandon Fosdick <b...@bfoz.net>
Subject Bucket brigades and large files
Date Fri, 28 Oct 2005 05:18:00 GMT
I'm still working on mod_dav_userdir, so naturally I have more questions.

Its passing most of the tests in litmus, with the only exceptions being some locking stuff
that neither windows nor osx seem to care about. Now I'm testing other stuff, namely large
file uploading. I have two large files that I'm working with, one is about 150MB and the other
is just over 4GB. My test client is OS X Tiger 10.4.2 and the server is 2.0.55 (Sempron 3000+
with 1GB of RAM).

With the 4GB file I eventually get the errors:

[Thu Oct 27 21:15:19 2005] [error] [client 192.168.0.1] Invalid Content-Length
[Thu Oct 27 21:15:19 2005] [error] [client 192.168.0.1] (34)Result too large: Could not get
next bucket brigade  [500, #0]

At this point nothing has been written to the database, even though the client has been transferring
for several minutes.

I can't find any mention of the bucket brigade error in the httpd source code, so I have no
idea what to do about it. Any ideas?

I have to admit I don't really understand how to interface with the bucket brigade stuff,
I blindly followed the example in mod_dav_fs. Even though the brigade handling appears to
break large files into pieces, it looks like the entire file is loaded before the repository
handler gets it. If that's the case, maybe the above errors are related to memory starvation?


FWIW, mod_dav_userdir, unlike catacomb, breaks the files into 64KB rows in the database. Consequently
I could easily benefit from being given the incoming bits in chunks, but I have no idea how
to accomplish that. Any suggestions?

During my travails I've noticed that the progress bars on the client machines (osx or win2k)
progress signifcantly faster than the rows are inserted into the database. I'm not entirely
sure what that means or what to do about it, but it would be nice to keep things in sync.

BTW, if any of you feel like pounding my server for me, feel free. The test server is at http://site46.com
(that's a reverse proxy to the real test server...let me know if it doesn't work). The real
server is partially at http://terran-bank.com. How was that for a shameless plug? :)

BTW2, if anyone cares I can make the source for mod_dav_userdir available. It's BSD, but I
haven't gotten around to posting it yet.

Thanks

Mime
View raw message