commons-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From sebb <seb...@gmail.com>
Subject Re: [net] Problem with completePendingCommand and large files > 600MB
Date Wed, 25 Oct 2017 22:57:15 GMT
What you are doing is stopping reading from the server before the file
has been fully sent.
Unless the server is expecting that, it may complain (the fact that it
works for smaller files may be because the server has already sent the
data).

AFAICT there is no standard way to tell the server to only send a
portion of the file.

A work-round would be to fetch all the data but stop writing it to
disk when you have got what you want.

Another way might be to cancel the transfer after enough data has been received.
However that relies on the server being able to process control
channel requests whilst sending data.

Or just live with the error message.

It's not a bug in NET.


On 25 October 2017 at 23:30, Bernd Eckenfels <ecki@zusammenkunft.net> wrote:
> 426 sounds like a server error, did you try to use a different client with your server?
>
> Gruss
> Bernd
> --
> http://bernd.eckenfels.net
> ________________________________
> From: SeungWook Kim <seungwook.kim@weather.com>
> Sent: Wednesday, October 25, 2017 6:34:02 PM
> To: user@commons.apache.org
> Subject: [net] Problem with completePendingCommand and large files > 600MB
>
> Hi,
>   I am having a problem with commons-net version* 3.6* and with large files
>> 600MB and offset and length that does not include the whole file.
>
> I have included the test code below that appears to fail with large files >
> 600 MB and an offset and length that does not include the whole file. If I
> choose a smaller file like 90KB with offset and length, it works
> successfully. If I choose a large file > 600MB with offset and length that
> INCLUDES the entire file, it also works successfully. The
> completePendingCommand replies back with "426 Failure writing network
> stream" when the file is large > 600MB AND the offset and length is not the
> entire file.
>
>   I believe this is a bug but not sure. I did a quick look in the postings
> and did not see anything that is exactly like it. Any feedback is greatly
> appreciated.
>
>
>
> *** Need to specify serverName, userName,...absOutputFile below***
> ***Start of Test code***
>
> public class FtpTest {
>     private static Logger log = LoggerFactory.getLogger(FtpTest.class);
>
>     private static final int DEFAULT_TIMEOUT = 10*1000;
>
>     @Before
>     public void setup() {
>
>     }
>
>     @Test
>     public void test() {
>         String serverName = "***";
>         String userName = "***";
>         String password = "***";
>         String absRemoteFileName = "***";
>         final long offset = ***;
>         final long length = ***;
>         String absOutputFile = "***";
>
>         FTPClient ftpClient = new FTPClient();
>         try {
>             ftpClient.connect(serverName);
>             int reply = ftpClient.getReplyCode();
>             if (!FTPReply.isPositiveCompletion(reply)) {
>                 String lastReply = ftpClient.getReplyString();
>                 fail(lastReply);
>             }
>             ftpClient.addProtocolCommandListener(new
> PrintCommandListener(new PrintWriter(System.out),
>                     true));
>             ftpClient.setKeepAlive(true);
>             ftpClient.setSoTimeout(DEFAULT_TIMEOUT); //set to 10 seconds
>             //turn on SO_LINGER w/ timeout of 0 to cause connection to be
> aborted immediately and to discard any
>             // pending data.
>             ftpClient.setSoLinger(true, 0);
>             if(ftpClient.login(userName, password)) {
>                 ftpClient.enterLocalPassiveMode();
>                 if(FTPReply.isPositiveCompletion(ftpClient.getReplyCode()))
> {
>                     log.info("Successfully connected in Passive Mode and
> logged into {}", serverName);
>
>                     ftpClient.setFileTransferMode(FTP.STREAM_TRANSFER_MODE);
>                     ftpClient.setFileType(FTP.BINARY_FILE_TYPE);
>                     ftpClient.setRestartOffset(offset);
>                     File writeToFile = new File(absOutputFile);
>
>                     try (InputStream inputStream =
> ftpClient.retrieveFileStream(absRemoteFileName);
>                          OutputStream outputStream =
> FileUtils.openOutputStream(writeToFile)) {
>                         if(inputStream == null || outputStream == null) {
>                             fail("One of the input/output stream is null");
>                         }
>                         final int BUFFER_SIZE = 4096;
>                         byte[] buffer = new byte[BUFFER_SIZE];
>                         int bytesRead;
>                         int len = (int)length;
>                         int bufferSize = length > BUFFER_SIZE ? BUFFER_SIZE
> : len;
>                         while ((bytesRead = inputStream.read(buffer, 0,
> bufferSize)) != -1) {
>                             len -= bytesRead;
>                             bufferSize = len > BUFFER_SIZE ? BUFFER_SIZE :
> len;
>
>                             outputStream.write(buffer, 0, bytesRead);
>
>                             if(len <= 0) {
>                                 break;
>                             }
>                         }
>
>
>                     } finally {
>                         if(ftpClient.completePendingCommand()) {
>                             log.info("Successful complete pending command");
>                         } else {
>                             fail("Unsuccessful complete pending command");
>                         }
>                     }
>
>                 } else {
>                     fail("Problem entering passive mode. All modern ftp
> server supports passive mode, but " +
>                             serverName + " doesn't??");
>
>                 }
>             } else {
>                 fail("Bad credentials to server " + serverName);
>             }
>
>         } catch (Exception e) {
>             fail(e.toString() + "\n" + e.getStackTrace().toString());
>         }
>     }
> }
>
> ***End of Test code***
>
> Thank you
> -Seung

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org


Mime
View raw message