camel-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alexander Lee (JIRA)" <j...@apache.org>
Subject [jira] Created: (CAMEL-1824) GenericFileConsumer fails to pick up all files in a directory if there are more files than maxMessagePerPoll setting
Date Sat, 11 Jul 2009 21:03:33 GMT
GenericFileConsumer fails to pick up all files in a directory if there are more files than
maxMessagePerPoll setting
--------------------------------------------------------------------------------------------------------------------

                 Key: CAMEL-1824
                 URL: https://issues.apache.org/activemq/browse/CAMEL-1824
             Project: Apache Camel
          Issue Type: Bug
          Components: camel-core
    Affects Versions: 2.0-M2
            Reporter: Alexander Lee


Camel adds filenames it is processing to an in memory, in progress map.  Once it is finished
processing it removes them.  If we specific a maxMessagesPerPoll number less then the files
in a directory, it only processes up to the number of files we specify for each poll.  It
then removes the rest from the in progress map and tries them again next poll.  This is the
code (from GenericFileConsumer:processBatch()) for the last part:

       for (int index = 0; index < exchanges.size() && isRunAllowed(); index++)
{
            GenericFileExchange<T> exchange = (GenericFileExchange<T>) exchanges.poll();
            String key = exchange.getGenericFile().getFileName();
            endpoint.getInProgressRepository().remove(key);
        }
 
Unfortunately, as you can see it uses exchanges.size() to determine how many file names to
remove (i.e. how many times to loop), however exchanges.poll() removes one from the head of
exchanges for each loop.  This means that the exchanges.size() reduces by one for each loop,
which means it only cleans up half of the filenames that are in exchanges, which means these
files are never picked up again as Camel thinks it is still processing them.

The fix is to replace the for loop with a while:

        while ((exchanges.size() > 0) && isRunAllowed()) {
            GenericFileExchange<T> exchange = (GenericFileExchange<T>) exchanges.poll();
            String key = exchange.getGenericFile().getFileName();
            endpoint.getInProgressRepository().remove(key);
        }




-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message