Mailing-List: contact dev-help@httpd.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@httpd.apache.org
Received-SPF: neutral (asf.osuosl.org: local policy)
Message-ID: <44492D20.3000004@bfoz.net>
Date: Fri, 21 Apr 2006 12:06:08 -0700
From: Brandon Fosdick <bfoz@bfoz.net>
User-Agent: Thunderbird 1.5 (X11/20060401)
MIME-Version: 1.0
To: dev@httpd.apache.org
Subject: Re: Large file support in 2.0.56?
References: 
 <F6E88684A95CDB4DBBD5B9F4C1C4B09B020EB20F@vis01-mx02.vis.internal.vodafone.com>
In-Reply-To: 
 <F6E88684A95CDB4DBBD5B9F4C1C4B09B020EB20F@vis01-mx02.vis.internal.vodafone.com>
Content-Type: text/plain; charset=iso-8859-1; format=flowed
Content-Transfer-Encoding: 8bit

Pl�m wrote:
> Have you checked if you can write the files with the default mod_dav_fs provider to
> the disk?

good suggestion, thanks...

Ok, same test setup that I posted about the other day, but this time I used mod_dav_fs. 

I'm getting slightly different behavior, in that the upload works in certain situations, but overall the same symptoms. If I don't cancel the transfer once the client says "server disconnected", it will eventually finish. I can get this behavior reliably with mod_dav_fs on 2.2.0 (haven't tried 2.0.55). Using my provider I get crashes on 2.0.55, but it eventually finishes on 2.2.0. So that's progress, but still not great.

It looks to me, and I could be wrong, that httpd/mod_dav is trying to buffer the entire request before giving it to the provider. The client, after finishing the transfer, is expecting a response from the server, but httpd doesn't send a response until the provider has finished with the request. Therefore, the response is delayed long enough that the client thinks the server has disappeared, when in fact the server is simply busy. Apparently the client (OS X in this case) is smart enough to handle the late response, so long as the user hasn't done the obvious thing and clicked the cancel button on the error box that pops up. If the user does click cancel, the client sends a stop request and httpd/mod_dav stops streaming the file to the provider, leaving an incomplete/corrupt file on disk.

Does that make sense to anyone? I don't know enough about the request handling process to know if I'm anywhere close on this one.

If my theory is correct, then I think the solution is to find a way to stream data to the storage provider earlier in the request process. I don't know if that's a core issue, or just some config bits in mod_dav, or my provider, that need to be fiddled. It's odd that httpd buffers the whole thing and then mod_dav streams it in 2K chunks, so I've got a feeling there's something in mod_dav that needs tweaking.