camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Claus Ibsen" ...@silverbullet.dk>
Subject RE: No way to remove FTP/SFTP files?
Date Fri, 06 Jun 2008 05:41:33 GMT
Hi Drew

Great work. Keep it up and I will later commit it into Camel.
When you think its ready then please submit it as a patch to CAMEL-570.

Camel does really need this feature to remove consumed FTP files. Transferring messages using
FTP is still a very common use case in real life.


Med venlig hilsen
 
Claus Ibsen
......................................
Silverbullet
Skovsgårdsvænget 21
8362 Hørning
Tlf. +45 2962 7576
Web: www.silverbullet.dk

-----Original Message-----
From: Drew McAuliffe [mailto:drewmca@gmail.com] 
Sent: 6. juni 2008 07:35
To: camel-user@activemq.apache.org
Subject: RE: No way to remove FTP/SFTP files?


Just as an FYI, I've hacked the following (after adding an "autoDelete"
property to the endpoint configuration. This is in "pollFile" in
FtpConsumer. It's a little staggered for debugging purposes, and I'm sure
the exception throwing isn't ideal, but it seems to work. This is based
somewhat on the Mule FTP and SFTP components. I'm working on the SFTP
version next. While I think a FileProcessStrategy approach similar to the
File component might be nice, for now an autodelete option works just fine
(I delete the files read from SFTP after moving them to local storage, where
I can handle them with proper backup after reading). Again, I think if
you're going to go through the effort of implementing some sort of
FileProcessStrategy thing, it would be better to do it once using a VFS
component instead of splitting it between FTP and SFTP because of the
different clients.

                getAsyncProcessor().process(exchange, new AsyncCallback(){
                	public void done(boolean sync) {
                        boolean failed = exchange.isFailed();
                        boolean handled =
DeadLetterChannel.isFailureHandled(exchange);
                        
                        if (LOG.isDebugEnabled()) {
                            LOG.debug("Done processing file: " + ftpFile +
". Status is: " + (failed ? "failed: " + failed + ", handled by failure
processor: " + handled : "OK"));
                        }

                        if (!failed || handled) {
                        	// if autodelete, then remove the source file
                        	if (endpoint.getConfiguration().isAutoDelete()){
                        		try {
                        			String toDelete = getFullFileName(ftpFile);
									boolean deleted = client.deleteFile(toDelete);
									if (! deleted)
										throw new IOException(MessageFormat.format("Could not delete
remote file at path {0}. Ftp error:{1}",
											new Object[]{ftpFile.getName(), client.getReplyCode()}));
									
								} catch (IOException e) {
									throw new RuntimeException(e.getMessage(), e);
								}
                        	}
                        } else if (failed && !handled) {
                            // there was an exception but it was not handled
by the DeadLetterChannel
                            handleException(exchange.getException());
                        }
                	}
                });
  


Claus Ibsen wrote:
> 
> Hi
> 
> Added ticket: CAMEL-570 to track this issue.
> 
> 
> Med venlig hilsen
>  
> Claus Ibsen
> ......................................
> Silverbullet
> Skovsgårdsvænget 21
> 8362 Hørning
> Tlf. +45 2962 7576
> Web: www.silverbullet.dk
> 
> -----Original Message-----
> From: Claus Ibsen [mailto:ci@silverbullet.dk] 
> Sent: 3. juni 2008 05:50
> To: camel-user@activemq.apache.org
> Subject: RE: No way to remove FTP/SFTP files?
> 
> Hi
> 
> And voting for the tickets also helps the team decide which issues we
> should tackle first.
> 
> Personally I do think file based components needs an overhaul in Camel to
> improve their standards as this is IMHO still one of the most common
> integration techniques in my daily fields of work.
> 
> Gert good point about that ServiceMix FTP component. Good spot for code
> resuse.
> 
> 
> 
> Med venlig hilsen
>  
> Claus Ibsen
> ......................................
> Silverbullet
> Skovsgårdsvænget 21
> 8362 Hørning
> Tlf. +45 2962 7576
> Web: www.silverbullet.dk
> -----Original Message-----
> From: Gert Vanthienen [mailto:gert.vanthienen@skynet.be] 
> Sent: 3. juni 2008 04:42
> To: camel-user@activemq.apache.org
> Subject: Re: No way to remove FTP/SFTP files?
> 
> Drew,
> 
> 
> No, you are not crazy.  Currently, the FTP/SFTP consumers don't support 
> this.  They store the lastPollTimestamp internally and compare that to 
> the file's timestamps to determine if a file still needs to be 
> processed.  However, the lastPollTime isn't being persisted anywhere, so 
> restarting the JVM will result in re-reading the files.
> 
> If you want, you can always provide a patch for the file-handling 
> strategies you want to use for your project.  Let us know if you need 
> any help with that...  Any (even partial) solution for this issue would 
> be more than welcome.
> 
> If you are using Camel inside ServiceMix, you can also use ServiceMix's 
> FTP poller component as a (temporary) workaround.  That one does already 
> support most forms of file-handling (delete, move, ...).
> 
> 
> Regards,
> 
> Gert
> 
> 
> 
> Drew McAuliffe wrote:
>> Am I crazy or is there no way that the FTP/SFTP consumers do anything to
>> a
>> source file once they've read it? There doesn't appear to be any delete
>> or
>> move option like with the File consumer. What possible use case would
>> that
>> support? Doesn't this pretty much guarantee that an FTP/SFTP consumer
>> will
>> continuously re-read files, never stopping?
>>
>> It appears that there's a JIRA issue to add support for something similar
>> to
>> a file renaming strategy, but it doesn't look like it's slated for
>> anything
>> in 1.4.
>>   
> 
> 
> 

-- 
View this message in context: http://www.nabble.com/No-way-to-remove-FTP-SFTP-files--tp17612896s22882p17684963.html
Sent from the Camel - Users mailing list archive at Nabble.com.


Mime
View raw message