camel-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alexander Lee (JIRA)" <>
Subject [jira] Commented: (CAMEL-1927) FileConsumer preMove setting causes filename to be left in in progress repository.
Date Sun, 23 Aug 2009 23:37:57 GMT


Alexander Lee commented on CAMEL-1927:

Hi Claus,

Haven't worked around it yet, not sure I know how it should be done as the code which sets
the filename in the in progress repository is in a separate place to the code that removes
it, and the rename strategy is invoked in between.  You could set the file name in the in
progress repository after the strategy is invoked but this might break something else.

I do like Camel, however I'm surprised how inheritance/composition based the components are
rather than following the pipeline architecture that Camel itself enables.  For instance renaming
could be a "processor" which the File component itself used as part of an initial pipeline.
 This would make the architecture more simple yet much more powerful.  So I would see the
File component being made up of a very simple timer with the second step as a file listing
processor, and so on, with the required functionality pipelined in.  Things like idempotent
or in progress repositories could be listeners to the Exchange object (for any changes in
name etc.), added at the start and removed on completion or error.  Settings on the File component
would then just decide whether a given processor was added to the initial flow/route or not.
 I would see it working something like this:

File Component (itself a Route/Flow - or the start of one)

Timer Endpoint -> File listing Processor -> Splitter (on filename) -> Seda/Direct
Queue -> In Progress Repository Processor -> Idempotent Repository Processor -> File
Renaming Processor -> File Reader Processor (into exchange body) -> etc.

So you would then be able to add functionality to the File component with less chance of breaking
something else as each processor would be decoupled from the next and each processor in a
component only has to concentrate on doing one thing well.  This would also allow the processors
to be reused by other developers and add functionality like the in progress or idempotent
repository to their own flows/routes.

This is another thing which sort of puzzles me is that there are a lot of endpoints out of
the box with Camel, but not many, if any processors.  If I want File renaming, deleting, moving
functionality as part of my flow/route, then I shouldn't need to write this myself.  At the
moment I'm trying to get around this by breaking my route into parts, so where I need file
renaming functionality I end one route, and use new route with a File endpoint to pick up
a file and rename it, and then from there another route to pick up the file again and do something
with it.  But this would all be simpler if there were file renaming processors, etc. out of
the box and this would add value to the Camel API.  In many cases it's as if I want some functionality
from an end point, but in the middle of my route/flow.  Of course I can and have written some
of these myself, but the next guy shouldn't have to.


> FileConsumer preMove setting causes filename to be left in in progress repository.
> ----------------------------------------------------------------------------------
>                 Key: CAMEL-1927
>                 URL:
>             Project: Apache Camel
>          Issue Type: Bug
>          Components: camel-core
>    Affects Versions: 2.0-M3
>            Reporter: Alexander Lee
>             Fix For: 2.1.0
> If the FileConsumer preMove setting is used, then files are picked up the first time
only, as the original file name is added to the in progress repository, whereas the new file
name is removed from the in progress repository (though nothing is actually removed as the
new file name is not in the in progress repository).
> This causes a few problems:
> i) If the maxMessagePerPoll is used, then messages not processed the first time are never
picked up again.
> ii) If the intention is to reprocess the same file, or a new file with the same name,
then it will never be picked up again after the first time.
> iii)  As the wrong file names are removed from the in progress repository, it results
in a slow memory leak in the in progress repository.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message