activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Antun Pendo (JIRA)" <j...@apache.org>
Subject [jira] Updated: (AMQ-2548) Downloading Blob messages via FTP fails for files larger than 64KB
Date Tue, 22 Dec 2009 18:51:40 GMT

     [ https://issues.apache.org/activemq/browse/AMQ-2548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Antun Pendo updated AMQ-2548:
-----------------------------

    Description: 
The following code will only download 64 KB of any uploaded file greater than 64 KB.
The test file (ca. 15 MB) was completly  uploaded to the FTP-Server.

 

{code}

    File file = new File(directoryName+fileName);
    
    ActiveMQConnectionFactory factory = new ActiveMQConnectionFactory(URI);
    Connection connection = factory.createQueueConnection(); 

    connection.start();
    
    ActiveMQSession session = (ActiveMQSession) connection.createSession(
                                false, Session.AUTO_ACKNOWLEDGE);
    Destination destination = session.createQueue("MyQ");
    MessageProducer producer = session.createProducer(destination);
    MessageConsumer consumer = session.createConsumer(destination);
    BlobMessage message = session.createBlobMessage(file);
      
    producer.send(message);

    System.out.println("Sent: " + message);
    
    Thread.sleep(1000);
      
    // check message sent
    Message msg = consumer.receive();
    
    BlobDownloadStrategy strategy = new FTPBlobDownloadStrategy();
      
    InputStream input = strategy.getInputStream((ActiveMQBlobMessage)msg);

    File f=new File(fileName);
    OutputStream out=new FileOutputStream(f);
    byte buf[]=new byte[1024];
    int len;
    
    while((len=input.read(buf))>0){
      out.write(buf,0,len);
    }
    out.close();
    input.close(); 

    System.out.println("Received: " + message);
    
{code}

After examining org.apache.activemq.blob.FTPBlobDownloadStrategy it seemed suspicious that
the FTPClient connection was destroyed before the input stream is processed.

{code}
public InputStream getInputStream(ActiveMQBlobMessage message) throws IOException, JMSException
{
        URL url = message.getURL();
        
        setUserInformation(url.getUserInfo());
        String connectUrl = url.getHost();
        int port = url.getPort() < 1 ? 21 : url.getPort();

        FTPClient ftp = new FTPClient();
        try {
        	ftp.connect(connectUrl, port);
        } catch(ConnectException e) {
        	throw new JMSException("Problem connecting the FTP-server");
        }
        
        if(!ftp.login(ftpUser, ftpPass)) {
        	ftp.quit();
            ftp.disconnect();
            throw new JMSException("Cant Authentificate to FTP-Server");
        }
        String path = url.getPath();
        String workingDir = path.substring(0, path.lastIndexOf("/"));
        String file = path.substring(path.lastIndexOf("/")+1);
        
        ftp.changeWorkingDirectory(workingDir);
        ftp.setFileType(FTPClient.BINARY_FILE_TYPE);
        InputStream input = ftp.retrieveFileStream(file);

        ftp.quit(); // really?
        ftp.disconnect(); // really?
        
        return input;
    }
{code}

After commenting those two last ftp calls, files larger than 64 KB were downloaded properly,
but this should of course not be the final solution. 
Any suggestions?

Cheers, Toni

  was:
The following code will only download 64 KB of any uploaded file greater than 64 KB.
The test file (ca. 15 MB) was completly  uploaded to the FTP-Server.

 

{code}

    File file = new File(directoryName+fileName);
    
    ActiveMQConnectionFactory factory = new ActiveMQConnectionFactory(URI);
    Connection connection = factory.createQueueConnection(); 

    connection.start();
    
    ActiveMQSession session = (ActiveMQSession) connection.createSession(
                                false, Session.AUTO_ACKNOWLEDGE);
    Destination destination = session.createQueue("MyQ");
    MessageProducer producer = session.createProducer(destination);
    MessageConsumer consumer = session.createConsumer(destination);
    BlobMessage message = session.createBlobMessage(file);
      
    producer.send(message);

    System.out.println("Sent: " + message);
    
    Thread.sleep(1000);
      
    // check message sent
    Message msg = consumer.receive();
    
    BlobDownloadStrategy strategy = new FTPBlobDownloadStrategy();
      
    InputStream input = strategy.getInputStream((ActiveMQBlobMessage)msg);

    File f=new File(fileName);
    OutputStream out=new FileOutputStream(f);
    byte buf[]=new byte[1024];
    int len;
    
    while((len=input.read(buf))>0){
      out.write(buf,0,len);
    }
    out.close();
    input.close(); 

    System.out.println("Received: " + message);
    
{code}

After examining org.apache.activemq.blob.DefaultBlobUploadStrategy it seemed suspicious that
the FTPClient connection was destroyed before the input stream is processed.

{code}
public InputStream getInputStream(ActiveMQBlobMessage message) throws IOException, JMSException
{
        URL url = message.getURL();
        
        setUserInformation(url.getUserInfo());
        String connectUrl = url.getHost();
        int port = url.getPort() < 1 ? 21 : url.getPort();

        FTPClient ftp = new FTPClient();
        try {
        	ftp.connect(connectUrl, port);
        } catch(ConnectException e) {
        	throw new JMSException("Problem connecting the FTP-server");
        }
        
        if(!ftp.login(ftpUser, ftpPass)) {
        	ftp.quit();
            ftp.disconnect();
            throw new JMSException("Cant Authentificate to FTP-Server");
        }
        String path = url.getPath();
        String workingDir = path.substring(0, path.lastIndexOf("/"));
        String file = path.substring(path.lastIndexOf("/")+1);
        
        ftp.changeWorkingDirectory(workingDir);
        ftp.setFileType(FTPClient.BINARY_FILE_TYPE);
        InputStream input = ftp.retrieveFileStream(file);

        ftp.quit(); // really?
        ftp.disconnect(); // really?
        
        return input;
    }
{code}

After commenting those two last ftp calls, files larger than 64 KB were downloaded properly,
but this should of course not be the final solution. 
Any suggestions?

Cheers, Toni


> Downloading Blob messages via FTP fails for files larger than 64KB
> ------------------------------------------------------------------
>
>                 Key: AMQ-2548
>                 URL: https://issues.apache.org/activemq/browse/AMQ-2548
>             Project: ActiveMQ
>          Issue Type: Bug
>    Affects Versions: 5.3.0
>         Environment: OS: Windows XP
>            Reporter: Antun Pendo
>            Priority: Minor
>
> The following code will only download 64 KB of any uploaded file greater than 64 KB.
> The test file (ca. 15 MB) was completly  uploaded to the FTP-Server.
>  
> {code}
>     File file = new File(directoryName+fileName);
>     
>     ActiveMQConnectionFactory factory = new ActiveMQConnectionFactory(URI);
>     Connection connection = factory.createQueueConnection(); 
>     connection.start();
>     
>     ActiveMQSession session = (ActiveMQSession) connection.createSession(
>                                 false, Session.AUTO_ACKNOWLEDGE);
>     Destination destination = session.createQueue("MyQ");
>     MessageProducer producer = session.createProducer(destination);
>     MessageConsumer consumer = session.createConsumer(destination);
>     BlobMessage message = session.createBlobMessage(file);
>       
>     producer.send(message);
>     System.out.println("Sent: " + message);
>     
>     Thread.sleep(1000);
>       
>     // check message sent
>     Message msg = consumer.receive();
>     
>     BlobDownloadStrategy strategy = new FTPBlobDownloadStrategy();
>       
>     InputStream input = strategy.getInputStream((ActiveMQBlobMessage)msg);
>     File f=new File(fileName);
>     OutputStream out=new FileOutputStream(f);
>     byte buf[]=new byte[1024];
>     int len;
>     
>     while((len=input.read(buf))>0){
>       out.write(buf,0,len);
>     }
>     out.close();
>     input.close(); 
>     System.out.println("Received: " + message);
>     
> {code}
> After examining org.apache.activemq.blob.FTPBlobDownloadStrategy it seemed suspicious
that the FTPClient connection was destroyed before the input stream is processed.
> {code}
> public InputStream getInputStream(ActiveMQBlobMessage message) throws IOException, JMSException
{
>         URL url = message.getURL();
>         
>         setUserInformation(url.getUserInfo());
>         String connectUrl = url.getHost();
>         int port = url.getPort() < 1 ? 21 : url.getPort();
>         FTPClient ftp = new FTPClient();
>         try {
>         	ftp.connect(connectUrl, port);
>         } catch(ConnectException e) {
>         	throw new JMSException("Problem connecting the FTP-server");
>         }
>         
>         if(!ftp.login(ftpUser, ftpPass)) {
>         	ftp.quit();
>             ftp.disconnect();
>             throw new JMSException("Cant Authentificate to FTP-Server");
>         }
>         String path = url.getPath();
>         String workingDir = path.substring(0, path.lastIndexOf("/"));
>         String file = path.substring(path.lastIndexOf("/")+1);
>         
>         ftp.changeWorkingDirectory(workingDir);
>         ftp.setFileType(FTPClient.BINARY_FILE_TYPE);
>         InputStream input = ftp.retrieveFileStream(file);
>         ftp.quit(); // really?
>         ftp.disconnect(); // really?
>         
>         return input;
>     }
> {code}
> After commenting those two last ftp calls, files larger than 64 KB were downloaded properly,
but this should of course not be the final solution. 
> Any suggestions?
> Cheers, Toni

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message