camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vincent Nonnenmacher <vincent.nonnenmac...@gmail.com>
Subject Re: File processing patterns
Date Tue, 26 Feb 2013 16:28:06 GMT
could you break the problem in two stage, one that just use a custom filter
on the file consumer, responsible for letting all files in a set goes into
a sub-folder on your filesystem (or the sftp one) only if all files are
presents.

And a second route, polling from the root of those sub-folder (on for each
set) and implement your 'transacted' like logic of processing.

But for the first stage, it implies that you have access to the file list
(as suggested in the first of your mentioned link).

A would like to know the end story of your question, as I give it some
thoughts and not find a very 'clever' solution ;-)



On Sun, Feb 24, 2013 at 6:47 PM, tms <tsattar.jobs@googlemail.com> wrote:

> Hi,
>
> I'm using Camel 2.10 with Spring XML (as much as possible).
>
> I have done a bit of reading on the EIP's. However I'm still not sure the
> best way of realizing the scenario I have:
>
> Files coming into SFTP server.
> Camel polling SFTP server
> Files are named in sets: 1A, 1B, 1C 2A, 2B, 2C, 3A, 3B, 3C....
> Process files of a set only if all corresponding files in set exists
> (A/B/C)
> If processing of a file in the set fails then move whole set (A, B, C) to
> ERROR
>
> Initially I had 3 routes defined with regex to for each file on a set
> (A/B/C).
> However that did not seem correct as they were being processed
> independently. In addition it caused a problem when another set was placed
> e.g. (1s and 2s)
>
> I then looked into the Aggregator pattern and also found the following
> post:
> http://camel.465427.n5.nabble.com/SFTP-Handshake-td475761.html
> <http://camel.465427.n5.nabble.com/SFTP-Handshake-td475761.html>
>
> Based on this I tried to implement my a custom GenericFileProcessStrategy
> but came across a few issues. The standard one returned by the built in
> factory method returns GenericFileProcessStrategy with renamers populated
> correctly. Seemed like I'd have to replicate most of the logic in the
> factory method to get mine working.
>
> Even if that is the case how would I process the other files? Do I
> dynamically create routes in Java DSL  with the expected filename as I
> would
> know the prefix (e.g. 1)?
>
> I also looked at the Message Router pattern where it routes file A to route
> A and file B to route B etc but again don't know best way to solve:
> 1.) only consume if the whole set is available
> and
> 2.) only proceed with next file in set if previous is successful.
>
> Thank you in advance.
>
>
>
> --
> View this message in context:
> http://camel.465427.n5.nabble.com/File-processing-patterns-tp5728066.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message