camel-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Claus Ibsen (JIRA)" <>
Subject [jira] [Assigned] (CAMEL-7521) Provide an option for unsynchronized aggregation when splitter is streaming and not parallel
Date Wed, 02 Jul 2014 07:54:24 GMT


Claus Ibsen reassigned CAMEL-7521:

    Assignee: Claus Ibsen

> Provide an option for unsynchronized aggregation when splitter is streaming and not parallel
> --------------------------------------------------------------------------------------------
>                 Key: CAMEL-7521
>                 URL:
>             Project: Camel
>          Issue Type: Improvement
>          Components: camel-core
>    Affects Versions: 2.13.2
>            Reporter: Jerry Williamson
>            Assignee: Claus Ibsen
>             Fix For: 2.14.0
>         Attachments:, camel.multicast.processor.patch
> We use a splitter route that splits very large files.
> The splits need to be aggregated in order.
> The splitter is therefore configured with streaming=true and parallelProcessing=false.
> When there are multiple in-flight exchanges on the route (i.e. different files are being
processed) the end-to-end processing time for each file is significantly impacted because
each aggregation call is synchronized in MulticastProcessor.doAggregate method..
> It is not uncommon, in our environment, to have thousands of splits per file and the
synchronization is significantly impacting the throughput of the route.
> I propose that an option be added (to splitter at least) to allow for an unsynchronized
aggregation. The option would default to false for backward compatibility and would possibly
be ignored if parallelProcessing = true.
> The body of the MulticastProcess.doAggregate method could be moved to a new, unsynchronized
method (say doAggregateInternal). The MulticastProcessor.doAggregate method could remain synchronized
(again for backward compatibility) but the body of that method would then call the new unsynchronized
doAggregateInternal method. In this way, existing code would remain synchronized.
> The splitter, then, could call the unsynchronized doAggregateInternal method when the
unsynchronizedAggregation option was true.

This message was sent by Atlassian JIRA

View raw message