camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vincent Nonnenmacher <vincent.nonnenmac...@gmail.com>
Subject Re: Advice on mirroring using FTP2 component
Date Mon, 03 Sep 2012 23:07:04 GMT
On Mon, Sep 3, 2012 at 9:32 PM, Tim Dudgeon <tdudgeon.ml@gmail.com> wrote:

> On 03/09/2012 19:54, Vincent Nonnenmacher wrote:
>
>> one way could be to not use idempotent and implement a filter that return
>> true or false on the modification date of the underlying file and handle a
>> water mark so you know that any file with such date after the mark is
>> selected.
>>
> Yes, maybe, but that means you need to stat the remote file first, not
> download it. AFAIK its completely different process to what I'm currently
> trying to do.
> Maybe it can be made to.


oh sorry, I thought your master repository was your local file system and
you ftp a copy (mirror)


>
>
>
>> On other could be to implement your own org.apache.camel.spi.**IdempotentRepository
>> object that would delegate to one of the current Memory or File based one,
>> but overrides the contains(key) to return false for any given entry that is
>> already present (processed) but which 'modification date' is after your
>> 'water mark'.
>>
> Sounds better. The IdempotentRepository would need to handle modification
> time and file size as well to do this properly, and the interface doesn't
> support this:
> http://camel.apache.org/maven/**current/camel-core/apidocs/**
> org/apache/camel/spi/**IdempotentRepository.html<http://camel.apache.org/maven/current/camel-core/apidocs/org/apache/camel/spi/IdempotentRepository.html>
> as it only takes a single param, and that's the file path. How does file
> size and modification time get passed?
>

yes you got a File so you can have its mod/creation date and stat the size




>
> It seems I'm being really stupid here, but it feels like I'm trying to
> re-invent a wheel that's been invented before plenty of times.
>
> Tim
>
>
>
>
>>
>>
>> On Mon, Sep 3, 2012 at 6:31 PM, Tim Dudgeon <tdudgeon.ml@gmail.com>
>> wrote:
>>
>>> Hi, I'm new to Camel, so please excuse my ignorance.
>>> I'm wanting to set up mirroring of files on a FTP server, so that
>>> whenever
>>> new files appear, or old files get changed, they get copied to the local
>>> mirror, and maybe some processing performed on them.
>>>
>>> I've worked out the basics of how to use the FTP and File components,
>>> and am
>>> now wondering how to actually set this up to do what I want. I got the
>>> idempotentRepository working (using a FileIdempotentRepository) to make
>>> sure
>>> that files are only processed once, but then realised that this isn't
>>> going to allow updated files to be re-fetched.
>>>
>>> Is there an already baked approach to doing this, or do I need to cook my
>>> own?
>>>
>>>
>>> Tim
>>>
>>>
>>>
>>> --
>>> View this message in context:
>>>
>> http://camel.465427.n5.nabble.**com/Advice-on-mirroring-using-**
>> FTP2-component-tp5718557.html<http://camel.465427.n5.nabble.com/Advice-on-mirroring-using-FTP2-component-tp5718557.html>
>>
>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message