camel-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (CAMEL-5828) Offer LMAX Disruptor pattern as an endpoint in CAMEL
Date Fri, 29 Mar 2013 01:21:17 GMT

    [ https://issues.apache.org/jira/browse/CAMEL-5828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13616941#comment-13616941
] 

Eric edited comment on CAMEL-5828 at 3/29/13 1:21 AM:
------------------------------------------------------

Great to see you in here.  I have integrated your code in my system, but I have seen a problem
with the exchange being copied when the consumer takes it from the ring buffer.  The unit
of work was modified by two threads, were one would make it null and the other would throw
a NPE.  To fix it, I made the copy before inserting in the ring buffer, by make a correlated
copy of the exchange.  I have a Unit Test to reproduce it on demand.

I also copied all the unit tests from the Seda component, and ported them to the disruptor
component.  I have ported about 30% of them so far, all tests passing.  I have also modified
the code base a little bit, making the Component in charge of creating the endpoints and managing
the different disruptor instances.  Just like the seda endpoints, you now can reference a
disruptor using a URI with or without the options:

From("disruptor:test?concurrentConsumers=3")
Will reference the same disruptor instance as
From("disruptor:test")

I haven't done extensive performance tests yet, so I can't comment on your multithread issues.

I am now at the stage of handling when a asynchronous request-reply is timing out, and removing
the exchange (on making the consumer ignore the exchange) from the ring buffer.  Maybe the
new version 3.0 would help, but I am not sure yet.

                
      was (Author: ladoe00):
    Great to see you in here.  I have integrated your code in my system, but I have seen a
problem with the exchange being copied when the consumer takes it from the ring buffer.  The
unit of work was modified by two threads, were one would make it null and the other would
throw a NPE.  To fix it, I made the copy before inserting in the ring buffer, by make a correlated
copy of the exchange.  I have a Unit Test to reproduce it on demand.

I also copied all the unit tests from the Seda component, and ported them to the disruptor
component.  I have ported about 30% of them so far, all tests passing.  I have also modified
the code base a little bit, making the Component in charge of creating the endpoints and managing
the different disruptor instances.  Just like the seda endpoints, you now can reference a
disruptor using a URI with or without the options:

From("disruptor:test?concurrentConsumers=3")
Will reference the same disruptor instance as
From("disruptor:test")

I haven't done extensive performance tests yet, so I can't comment on your multithread issues
yet.

I am now at the stage of handling when a asynchronous request-reply is timing out, and removing
the exchange (on making the consumer ignore the exchange) from the ring buffer.  Maybe the
new version 3.0 would help, but I am not sure yet.

                  
> Offer LMAX Disruptor pattern as an endpoint in CAMEL
> ----------------------------------------------------
>
>                 Key: CAMEL-5828
>                 URL: https://issues.apache.org/jira/browse/CAMEL-5828
>             Project: Camel
>          Issue Type: New Feature
>            Reporter: Eric
>             Fix For: Future
>
>
> My system is an hybrid of CAMEL and LMAX Disruptor, where threads consuming from LMAX
ringbuffer are feeding a "pool" of SEDA queues in a round-robin fashion.  This system operates
at very high-speed such as consumers from the SEDA queues and their producers are constantly
blocking on the queues.  Having a non-blocking, high-performance endpoint like LMAX Disruptor
would be a great addition to CAMEL.
> LMAX Disruptor website: http://lmax-exchange.github.com/disruptor/

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message