camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Claus Ibsen <>
Subject Re: Howto use maxMessagesPerPoll and sortBy for file-consumer
Date Thu, 19 Jan 2012 11:01:48 GMT
On Thu, Jan 19, 2012 at 11:57 AM, Hartmut Lang
<> wrote:
> Hi,
> what i see from is that maxMessagesPerPoll
and sortBy option do not work well together for a FileConsumer endpoint.
> In our use-case we have several thousand files in a from("file:...") route. And every
minute we want to process one file sorted by name.
> So we use the options: delay=60000&maxMessagesPerPoll=1&sortBy=file:name
> But the way it seems to be implemented is:
> - read one file from fileystem
> - sort this file by name (so nothing happens here)
> - process this file
> The sortBy does not have any effect here, right?

Yes it used to sort before limiting. But then we had people with
1000000's of files, which took to long time to sort, as the sorting
was in-memory. And the FTP component re-uses the logic, and the ftp
file entries eat up more memory, so that was even slower.

So the file component will limit first, then sort afterwards.

> Can someone propose another solution where we can have the files processed sorted by-name
and also control how fast they are processed?
> Thanks,
> Hartmut

Claus Ibsen
Twitter: davsclaus, fusenews
Author of Camel in Action:

View raw message