camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From wmoussel <wmous...@gmail.com>
Subject File Consumer with dynamic list of files to poll
Date Mon, 22 Feb 2010 08:04:52 GMT

Hi,

Here's my initial issue. I have to poll for new files in a directory
containing a lot of files (5000 to 20000) (and I can't move them after
processing).

When using idempotent option it takes way too much cpu (like 9% even when
they aren't new files).

So i tried adding a filter so that the consumer would only take files less
than 2  days old. It doesn't change much in the end. When I looked into how
filter works in Java I got why it's still cpu consuming...

So then I thought i would get recent file names with a unix command like
find . -mtime -1 and use pollenrich to poll specific files. The way I
understand how poll enrich works, it will poll the whole directory and then
try to match filename afterwards. That is also cpu consuming since it gets
all the files as exchanges prior to the match. Am I wrong ?

So I'm basically trying to invoke a FileName to GenericFileExchange
processor in the middle of my route. Is there a way to do this?

I'm open to any suggestion.

Thanks in advance :)

Wandrille
-- 
View this message in context: http://old.nabble.com/File-Consumer-with-dynamic-list-of-files-to-poll-tp27683938p27683938.html
Sent from the Camel - Users mailing list archive at Nabble.com.


Mime
View raw message