commons-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Martin Cooper" <mart...@apache.org>
Subject Re: [FileUpload] Large file uploads are slow
Date Sat, 11 Mar 2006 17:59:14 GMT
On 3/11/06, C Rose <junk5@microserf.org.uk> wrote:
>
> Hi again
>
> >> I am using a DiskFileItemFactory, and the only thing I do to it is
> >> mandate that the ServletFileUpload object created by the factory does
> >> not have a limit on the size of the file being uploaded. The
> >> transfer is
> >> going via HTTPS.
> >
> > I'm a little confused by this.
>
> [Snip]
>
> Sorry, it was Friday and I was a little fried. He's a snippet:
>
>
>                  DiskFileItemFactory factory = new DiskFileItemFactory
> ();
>                  ServletFileUpload upload = new ServletFileUpload
> (factory);
>                  upload.setSizeMax(-1); // No maximum upload size.
>                  List fileItems = upload.parseRequest(request);
>
> Later I use FileItem.write(File) to write the data to disk.


OK. Nothing wrong with that.

> First of all, the ServletFileUpload instance
> > is created by you, not by the DiskFileItemFactory. That factory
> > creates
> > FileItem instances. Second, an unlimited file size is the default,
> > so you
> > shouldn't be configuring anything if that's what you want.
>
> I only set the maximum size manually to make it explicit in the code
> what is intended, and to protect myself against changes to the API
> defaults in future versions (which I hope would be unlikely).


Right. I hope that's unlikely too. ;-)

As I said in my original email, it's parseRequest() that takes a long
> time to complete---and it'd be great to find a way to speed it up.


The parseRequest() method is where all the real work in FileUpload is. That
method has to dredge through your 45MB to find the beginning and end of each
form field, and stream the content to memory and then to disk (once the
threshold is reached). Unfortunately, there is very little you can do (with
respect to Commons FileUpload) that would affect the performance of this.

I suppose you could experiment with the threshold value. The default is
10KB, which means that a field's data will be saved in memory until it
reaches 10KB in size, at which point that and all subsequent data will be
written directly to disk. If you decreased that size, you would skip the
saving-in-memory part; if you increased it, you would write a bigger initial
chunk to disk. In the greater scheme of things, I wouldn't expect this to
make much of a difference over a 40MB upload, but it may depend on the
characteristics of your machine (VM size, disk speed, etc.).

Sorry I can't be of more help.

--
Martin Cooper


Thanks again
>
> C
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: commons-user-help@jakarta.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message