camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Claus Ibsen" ...@silverbullet.dk>
Subject RE: CAMEL-250: Still having probs
Date Sat, 26 Jul 2008 14:58:49 GMT
Hi

I have made some changes in Camel 1.5 that should fix Camel file consumer not reading files
that is in progress of being written.

If anyone feel like it, please test the Camel 1.5-SNAPSHOT.

The issue is tracked in CAMEL-760

And the same change has been made to the FTP consumer so it wont poll files "in progress of
being written". See CAMEL-654



Med venlig hilsen
 
Claus Ibsen
......................................
Silverbullet
Skovsgårdsvænget 21
8362 Hørning
Tlf. +45 2962 7576
Web: www.silverbullet.dk

-----Original Message-----
From: Claus Ibsen [mailto:ci@silverbullet.dk] 
Sent: 2. juli 2008 13:13
To: camel-user@activemq.apache.org
Subject: RE: CAMEL-250: Still having probs

Hi

Thanks for sharing this. Is handy for the work on 1.5 where we hopefully addresses this.


Med venlig hilsen
 
Claus Ibsen
......................................
Silverbullet
Skovsgårdsvænget 21
8362 Hørning
Tlf. +45 2962 7576
Web: www.silverbullet.dk

-----Original Message-----
From: borgel [mailto:borge.lotre@gmail.com] 
Sent: 2. juli 2008 13:11
To: camel-user@activemq.apache.org
Subject: RE: CAMEL-250: Still having probs


I had the same problem, so I created a Processor that waits until the file is
ready.
In my case I changed the route: 
from("file:c:/temp")
.to("ftp://camel@localhost?password=xxx&binary=true");	

to this:
from("file:c:/temp")
.process(waitProcessor)		
.to("ftp://camel@localhost?password=xxx&binary=true");	

The waitProcessor runs unit it is able to lock the file, like this:
FileChannel channel new RandomAccessFile(file, "rw").getChannel();
lock = channel.tryLock();

When lock is not null and not throws an error the method can return.


Claus Ibsen wrote:
> 
> Hi Law
> 
> The file components had some improvements from 1.3 to 1.4. As you write
> the lock file is removed, even if the poll failed.
> 
> In Camel 1.5 we have planned some improvements to file and ftp component.
> I do think this one http://issues.apache.org/activemq/browse/CAMEL-654
> also applies for the File component as well = there needs to be some sort
> of "progress" strategy.
> 
> When you copy the file, a strategy also common used is to copy it with a
> different name, and when the copying has finished you rename the file to
> the correct filename. It is then important to instruct Camel to exclude
> reading "in progress" files. Camel has options to set *.work etc. as
> excluded pattern.
> 
> 1) So you copy the file as: mybigfile.txt.work
> 2) And set Camel to exclude *.work
> 3) And when finished copying the file, then you rename mybigfile.txt.work
> to mybigfile.txt
> 
> 
> 
> 
> Med venlig hilsen
>  
> Claus Ibsen
> ......................................
> Silverbullet
> Skovsgårdsvænget 21
> 8362 Hørning
> Tlf. +45 2962 7576
> Web: www.silverbullet.dk
> 
> -----Original Message-----
> From: law [mailto:robert.law@americanwell.com] 
> Sent: 1. juli 2008 21:30
> To: camel-user@activemq.apache.org
> Subject: Re: CAMEL-250: Still having probs
> 
> 
> Looks like the 1.4 snapshot at least doesn't die upon having problems
> renaming a file. However, it will not re-process the file automagically,
> but
> will if I just rename it.
> 
> 
> law wrote:
>> 
>> The problem I am having is that when I copy a large file into the
>> directory that camel is reading from and it hasn't finished copying,
>> camel
>> attempts to copy the file into the .camel dir which fails since the
>> copying of the file is locking it since it hasn't finished. I tried to
>> use
>> the noop=true but that doesn't seem to pick up new files put in the
>> directory (I really don't know what's going on w/ that). However, I was
>> browsing the bugs and found:
>> 
>>  https://issues.apache.org/activemq/browse/CAMEL-250
>> https://issues.apache.org/activemq/browse/CAMEL-250  
>> 
>> which seems to indicate that the problem was fixed in 1.3, but I'm still
>> seeing it. Does anyone have any insights of what I can do to fix either:
>> 
>> a) The noop flag for files
>> b) Ignore files until they are ready to be read, then fire off the
>> consumer
>> c) If it is going to fail, how can I make it reprocess the file. It seems
>> to kill camel when this happens.
>> 
>> The setup I have is using spring bean for the .to() and having spring
>> start up the context.
>> 
> 
> -- 
> View this message in context:
> http://www.nabble.com/CAMEL-250%3A-Still-having-probs-tp18223318s22882p18223545.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
> 
> 
> 

-- 
View this message in context: http://www.nabble.com/CAMEL-250%3A-Still-having-probs-tp18223318s22882p18235156.html
Sent from the Camel - Users mailing list archive at Nabble.com.


Mime
View raw message