commons-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bernd Eckenfels (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (NET-550) Default FTPClient bufferSize results in very slow retrieve transfers
Date Thu, 07 Aug 2014 19:08:13 GMT

    [ https://issues.apache.org/jira/browse/NET-550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14089627#comment-14089627
] 

Bernd Eckenfels commented on NET-550:
-------------------------------------

Hm, I was looking at the usage of this in VFS, because it is affected: it does not set an
explicite buffer size. 

When I drill down into the 3.3 code, it looks like it correctly generated getBufferedXXXStream()
for __bufferSize==0 (using default buffer size of 8k from Java bufferedxxxStream). However
besides the buffered streams it also uses the buffer size for the copy loop. And in this case
it does not use a default buffer. Which is quite strange because I wonder why read[byte[0])
works anywy. It only works because it has a read() fallback, which is clearly not the intention
of the code.

Attached is a proposed minimal fix, but instead of that the buffering should be reconsidered
completely.

> Default FTPClient bufferSize results in very slow retrieve transfers
> --------------------------------------------------------------------
>
>                 Key: NET-550
>                 URL: https://issues.apache.org/jira/browse/NET-550
>             Project: Commons Net
>          Issue Type: Bug
>    Affects Versions: 3.3
>            Reporter: Geoffrey Hardy
>
> While experimenting with FTPClient, I discovered that if I don't call setBufferSize(),
the default value is zero.  This results in retrieveFile() calling the version of InputStream.read()
with no parameters, reading one byte at a time.  For comparison, the downloading a CD ISO
image of about ~648MB took 18m10s with the default settings.  In contrast, calling setBufferSize(8192)
took only 7.9s, an improvement of ~137x.
> Here is some sample code:
> {code:java}
> FTPClient ftp = new FTPClient();
> // ftp.setBufferSize(8192);
> ftp.setControlKeepAliveTimeout(300);
> ftp.setCopyStreamListener(new CopyStreamListener() {
>     @Override
>     public void bytesTransferred(long totalBytesTransferred, int bytesTransferred, long
streamSize) {
>         System.out.println("totalBytesTransferred: " + totalBytesTransferred
>             + ", bytesTransferred: " + bytesTransferred + ", streamSize: " + streamSize);
>     }
>     @Override public void bytesTransferred(CopyStreamEvent event) {}
> });
> ftp.connect(host);
> ftp.login(user, pass);
> ftp.retrieveFile(file, outputStream);
> {code}
> The log message from the stream listener printed lots messages "bytesTransferred: 1"
and totalBytesTransferred incremented by 1 each time.  This corresponds to the part of the
code which reads one byte at a time with {{int inputStream.read()}}.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message