commons-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nick Didkovsky <didk...@mail.rockefeller.edu>
Subject Re: FileUpload, is upload cached before FileUpload parses it?
Date Fri, 24 Mar 2006 19:21:20 GMT
Thank-you very much Martin

At your suggestion, I've posted this question to the tomcat users  
list and will report on what I discover

Re. #1, I don't mind some buffering, but my worry is that the input  
stream is being forked to FileUpload, so that in fact the file is  
going to two places simultaneously: both to FileUpload and to some  
other disk or memory on the server, perhaps resources allocated for  
Apache uploads.
Re. #2,  I can live with the oddity of file upload continuing after  
the exception is thrown as long as the incoming stream is not being  
cached or copied to memory or disk anywhere.  If after the exception  
is thrown, the file input stream is simply arriving and being  
ignored, then it would be merely inconvenient.  If it's coming in and  
writing to some temp drive or filling up memory, that would be bad.

Will let you know what more I find out.

Best regards
Nick Didkovsky

On Mar 24, 2006, at 1:18 PM, Martin Cooper wrote:

> On 3/24/06, Nick Didkovsky <didkovn@mail.rockefeller.edu> wrote:
>>
>> Hello
>>
>> I am using Commons FileUpload 1.1 to manage HTTP POST uploads with
>> Tomcat running as a JkMount'ed service of Apache
>>
>> I am trying to analyze the strain on memory when many large files are
>> being uploaded from different clients simultaneously
>>
>> I have two questions:
>> 1) Before a POST'ed file upload reaches FileUpload, is all or part of
>> it already cached somewhere on the server's drive space or memory, or
>> does it arrive from the network directly to FileUpload for parsing?
>> The latter would be desirable, as each uploaded file would get
>> written directly to the temp directory as it arrives over the
>> network, with no additional overhead.  Can someone verify/amend/
>> provide more details?
>
>
> You'd need to ask the Tomcat folks about this. FileUpload will  
> start parsing
> the input stream as soon as parseRequest() is invoked (and as soon  
> as it
> gets data from the request input stream). However, exactly when the
> container (e.g. Tomcat) invokes your servlet or JSP, and whether or  
> not it
> buffers content, and how much of it, would be up to the container.
>
> 2) I tried uploading a file whose size exceeds the max file size I
>> set in my test JSP page. The JSP handles the upload with FileUpload,
>> and writes a log file of activity wth time-stamps. Checking the log,
>> I see that my page throws the SizeLimitExceededException almost
>> immediately after the client's POST reaches the server.  Yet the
>> client browser sits there and waits for roughly the amount of time a
>> full upload would take, and only then does the browser display the
>> SizeLimitExceededException. The wait is longer for larger files.  Can
>> someone explain exactly what is happening during this time interval
>> between the exception being thrown on the server and the exception
>> appearing on the client?  Is the file actually being uploaded
>> somewhere to the server after the SizeLimitExceededException is
>> thrown? If not what then is going on during this minute or two? And
>> if so, where is the upload going and whose memory or disk resources
>> are being utilized? Note that FileUpload's temp directory does not
>> show any temp file growing when this exception is thrown.  Yet there
>> does seem to be some network activity between the client and the
>> server, but I am baffled as to what it could be and what its
>> implications might be.
>
>
> The problem is that, to my knowledge, there is no way for  
> FileUpload to tell
> the container to abandon the request. FileUpload throws the exception
> immediately because it is based on the content length header, so it  
> doesn't
> even try to read the input stream. However, the container continues to
> accept the request being sent from the browser, even when nobody is  
> going to
> read it.
>
> I haven't tried it, but you could try throwing an IOException out  
> of the
> servlet's service() method, and see if that will stop the container  
> from
> accepting data. Not that that's really a clean solution.
>
> If you do get helpful answers to these questions from the Tomcat  
> folks, I'd
> be very interested in hearing them.
>
> --
> Martin Cooper
>
>
> Thank-you,
>> Nick Didkovsky
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
>> For additional commands, e-mail: commons-user-help@jakarta.apache.org
>>
>>


---------------------------------------------------------------------
To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-user-help@jakarta.apache.org


Mime
View raw message