commons-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bernd Eckenfels <e...@zusammenkunft.net>
Subject Re: [net] Problem with completePendingCommand and large files > 600MB
Date Wed, 25 Oct 2017 22:30:24 GMT
426 sounds like a server error, did you try to use a different client with your server?

Gruss
Bernd
--
http://bernd.eckenfels.net
________________________________
From: SeungWook Kim <seungwook.kim@weather.com>
Sent: Wednesday, October 25, 2017 6:34:02 PM
To: user@commons.apache.org
Subject: [net] Problem with completePendingCommand and large files > 600MB

Hi,
  I am having a problem with commons-net version* 3.6* and with large files
> 600MB and offset and length that does not include the whole file.

I have included the test code below that appears to fail with large files >
600 MB and an offset and length that does not include the whole file. If I
choose a smaller file like 90KB with offset and length, it works
successfully. If I choose a large file > 600MB with offset and length that
INCLUDES the entire file, it also works successfully. The
completePendingCommand replies back with "426 Failure writing network
stream" when the file is large > 600MB AND the offset and length is not the
entire file.

  I believe this is a bug but not sure. I did a quick look in the postings
and did not see anything that is exactly like it. Any feedback is greatly
appreciated.



*** Need to specify serverName, userName,...absOutputFile below***
***Start of Test code***

public class FtpTest {
    private static Logger log = LoggerFactory.getLogger(FtpTest.class);

    private static final int DEFAULT_TIMEOUT = 10*1000;

    @Before
    public void setup() {

    }

    @Test
    public void test() {
        String serverName = "***";
        String userName = "***";
        String password = "***";
        String absRemoteFileName = "***";
        final long offset = ***;
        final long length = ***;
        String absOutputFile = "***";

        FTPClient ftpClient = new FTPClient();
        try {
            ftpClient.connect(serverName);
            int reply = ftpClient.getReplyCode();
            if (!FTPReply.isPositiveCompletion(reply)) {
                String lastReply = ftpClient.getReplyString();
                fail(lastReply);
            }
            ftpClient.addProtocolCommandListener(new
PrintCommandListener(new PrintWriter(System.out),
                    true));
            ftpClient.setKeepAlive(true);
            ftpClient.setSoTimeout(DEFAULT_TIMEOUT); //set to 10 seconds
            //turn on SO_LINGER w/ timeout of 0 to cause connection to be
aborted immediately and to discard any
            // pending data.
            ftpClient.setSoLinger(true, 0);
            if(ftpClient.login(userName, password)) {
                ftpClient.enterLocalPassiveMode();
                if(FTPReply.isPositiveCompletion(ftpClient.getReplyCode()))
{
                    log.info("Successfully connected in Passive Mode and
logged into {}", serverName);

                    ftpClient.setFileTransferMode(FTP.STREAM_TRANSFER_MODE);
                    ftpClient.setFileType(FTP.BINARY_FILE_TYPE);
                    ftpClient.setRestartOffset(offset);
                    File writeToFile = new File(absOutputFile);

                    try (InputStream inputStream =
ftpClient.retrieveFileStream(absRemoteFileName);
                         OutputStream outputStream =
FileUtils.openOutputStream(writeToFile)) {
                        if(inputStream == null || outputStream == null) {
                            fail("One of the input/output stream is null");
                        }
                        final int BUFFER_SIZE = 4096;
                        byte[] buffer = new byte[BUFFER_SIZE];
                        int bytesRead;
                        int len = (int)length;
                        int bufferSize = length > BUFFER_SIZE ? BUFFER_SIZE
: len;
                        while ((bytesRead = inputStream.read(buffer, 0,
bufferSize)) != -1) {
                            len -= bytesRead;
                            bufferSize = len > BUFFER_SIZE ? BUFFER_SIZE :
len;

                            outputStream.write(buffer, 0, bytesRead);

                            if(len <= 0) {
                                break;
                            }
                        }


                    } finally {
                        if(ftpClient.completePendingCommand()) {
                            log.info("Successful complete pending command");
                        } else {
                            fail("Unsuccessful complete pending command");
                        }
                    }

                } else {
                    fail("Problem entering passive mode. All modern ftp
server supports passive mode, but " +
                            serverName + " doesn't??");

                }
            } else {
                fail("Bad credentials to server " + serverName);
            }

        } catch (Exception e) {
            fail(e.toString() + "\n" + e.getStackTrace().toString());
        }
    }
}

***End of Test code***

Thank you
-Seung

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message