camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hartmut Lang <>
Subject RE: Howto use maxMessagesPerPoll and sortBy for file-consumer
Date Thu, 19 Jan 2012 11:07:44 GMT

maybe it's a good idea to add this to the documentation for file2, "Sorting using sortBy"

Hartmut Lang

-----Original Message-----
From: Claus Ibsen [] 
Sent: Donnerstag, 19. Januar 2012 12:02
Subject: Re: Howto use maxMessagesPerPoll and sortBy for file-consumer

On Thu, Jan 19, 2012 at 11:57 AM, Hartmut Lang <> wrote:
> Hi,
> what i see from is that maxMessagesPerPoll
and sortBy option do not work well together for a FileConsumer endpoint.
> In our use-case we have several thousand files in a from("file:...") route. And every
minute we want to process one file sorted by name.
> So we use the options: 
> delay=60000&maxMessagesPerPoll=1&sortBy=file:name
> But the way it seems to be implemented is:
> - read one file from fileystem
> - sort this file by name (so nothing happens here)
> - process this file
> The sortBy does not have any effect here, right?

Yes it used to sort before limiting. But then we had people with 1000000's of files, which
took to long time to sort, as the sorting was in-memory. And the FTP component re-uses the
logic, and the ftp file entries eat up more memory, so that was even slower.

So the file component will limit first, then sort afterwards.

> Can someone propose another solution where we can have the files processed sorted by-name
and also control how fast they are processed?
> Thanks,
> Hartmut

Claus Ibsen
Twitter: davsclaus, fusenews
Author of Camel in Action:

View raw message