commons-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Martin Cooper" <mart...@apache.org>
Subject Re: [FileUpload] Speeding up uploads
Date Thu, 12 Jun 2003 08:18:38 GMT

"Rademacher Tobias" <Tobias.Rademacher@grob.de> wrote in message
news:1D72EFB14F7DD611A7AF0003470DED25A5DBF2@exchmin01.gm.grob...
> Dear Martin,
>
> > > Wouldn't NIO come with some performance improvements
> > especially for large
> > > files?
> >
> > I don't know a whole lot about NIO, but I doubt that it would make any
> > difference. Since a given request is guaranteed to be
> > serviced entirely by a
> > single thread, there's not much opportunity for getting
> > clever. But then
> > again, even if there was, that's part of the container, and
> > not something
> > Commons FileUpload could do anything about.
> >
>
> As NIO works directly on OS Pipes I used it in order to get around on the
> byte buffer
> in-VM memory problem. It's fairly easy to wrap ServletOutputStream with
NIO
> Channels:
>
> /**
> * Writes the file from server to client using NIO over HTTP.
> *
> * @param out stream from tomcat to client browser
> * @param fileName the file to write over the wire
> * @exception FileNotFoundException if the request file does not
> exists
> * @exception IOException if the sending process fails
> */
> private void sendFile(ServletOutputStream out, String fileName)
> throws FileNotFoundException, IOException {
>
> RandomAccessFile file = null;
> FileChannel fileChannel = null;
> WritableByteChannel servletOutChannel = null;
> MappedByteBuffer buffer = null;
>
> try {
> file = new RandomAccessFile(fileName, "r");
> fileChannel = file.getChannel();
>
> servletOutChannel = Channels.newChannel(out);
>
> LOGGER.debug("Begin to write out:" +
> fileChannel.size());
>
> fileChannel.transferTo(0L, fileChannel.size(),
> servletOutChannel);
>
> } finally {
> if (buffer != null)
> buffer = null;
> if (fileChannel != null)
> fileChannel.close();
> if (servletOutChannel != null)
> servletOutChannel.close();
> if (file != null)
> file.close();
> }
> }

That might be fine for file *download*. ;-)

>
> Using this approach vice versa (on InputStreams may) may work as well into
a
> MulitPartStream using
> java.nio.ByteBuffer and Channels to transfer byte bulks. (Unfortunatly
this
> works only on
> 1.4+ which is not very portable.) I actually begin porting your module to
> NIO but as I don't
> understand _really_ what going on there (not the NIO stuff but the RFC is
> not really a good guide) so if you interested into that clumsy code, fell
> free to ask me for a copy. The main benifit of NIO for Upload would be
that
> you don't need to configure a max file size any more due to complete
> transfer could be managed outside the JVM head memory.

The maximum size configuration has nothing to do with memory usage. In the
default implementation, the entire upload is almost never retained in
memory, unless it is very small. Once it passes a configurable threshold, it
is written to disk instead.

The main reason for the maximum size configuration is so that clients are
constrained in how much data they can send to the server, which also helps
to prevent DoS attacks. Do you really want some random hacker uploading a
100GB file to your web server? ;-)

--
Martin Cooper


>
> Bye
> Toby




Mime
View raw message