camel-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ioannis Alexandrakis (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (CAMEL-5719) ZooKeeper route policy should initialize using onInit to elect master
Date Thu, 04 Dec 2014 09:02:12 GMT

    [ https://issues.apache.org/jira/browse/CAMEL-5719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14233674#comment-14233674
] 

Ioannis Alexandrakis edited comment on CAMEL-5719 at 12/4/14 9:01 AM:
----------------------------------------------------------------------

There is another issue I wanted to address, which applies here (and in the original component,
too). Perhaps it deserves its own ticket, perhaps not, perhaps it does not deserve a ticket
at all.

Following the above example, if someone has, for instance, activemq components, just stopping
the consumers (edit: the endpoint's consumers, not the activemq consumers) is not 'safe' enough.
I noticed that (prefetch size * consumer count) messages (I had 2 consumers with prefetch
1, so, all in all, 2 messages) are neither being processed, not forwarded further. They stay
in 'inflight' mode.

My setup is a store and forward network of activemq brokers. So, a queue on node A, if it
does not have any (activemq) consumers, it forwards to a node B which has. In my case, the
elected as master node would have to get the activemq messages, even though I sent jms messages
to node A, which was not elected from zookeeper. Instead, it skipped (prefetch size * consumer
count) messages before the rest got forwarded (the skipped ones stayed in node B as 'inflight',
apparently it awaits for the endpoint consumers to be started). If, instead of stopping the
consumers, I was to stop the route itself, then the component did not get those messages as
inflight and all messages got forwarded immediately. I have a patch available on top of the
above one (it just stops the route where it stops the consumers of the endpoint, and starts
the route wherever it starts the consumers. The idea is the same, but instead of consumers,
the whole route gets started/stoped). I can attach it, if you like.

This is a matter of opinion, really. If someone sets the prefetch size = 0, then there is
no problem. However, it would drop the performance significantly. In my opinion, however (in
any case), I think the zookeeper policy should 'stop' the route entirely. I do not know if
such a behaviour exists in other components (like jms/activemq's prefetch size and the inflight
exchanges problem). So, perhaps this is a really unique/specific case.


was (Author: ialex):
There is another issue I wanted to address, which applies here (and in the original component,
too). Perhaps it deserves its own ticket, perhaps not, perhaps it does not deserve a ticket
at all.

Following the above example, if someone has, for instance, activemq components, just stopping
the consumers is not 'safe' enough. I noticed that (prefetch size * consumer count) messages
(I had 2 consumers with prefetch 1, so, all in all, 2 messages) are neither being processed,
not forwarded further. They stay in 'inflight' mode.

My setup is a store and forward network of activemq brokers. So, a queue on node A, if it
does not have any consumers, it forwards to a node B which has. In my case, the elected as
master node would have to get the activemq messages, even though I sent jms messages to node
A, which was not elected from zookeeper. Instead, it skipped (prefetch size * consumer count)
messages before the rest got forwarded (the skipped ones stayed in node B as 'inflight').
If, instead of stopping the consumers, I was to stop the route itself, then the component
did not get those messages as inflight and all messages got forwarded immediately. I have
a patch available on top of the above one (it just stops the route where it stops the consumers
of the endpoint, and starts the route wherever it starts the consumers. The idea is the same,
but instead of consumers, the whole route gets started/stoped). I can attach it, if you like.

This is a matter of opinion, really. If someone sets the prefetch size = 0, then there is
no problem. However, it would drop the performance significantly. In my opinion, however (in
any case), I think the zookeeper policy should 'stop' the route entirely. I do not know if
such a behaviour exists in other components (like jms/activemq's prefetch size and the inflight
exchanges problem). So, perhaps this is a really unique/specific case.

> ZooKeeper route policy should initialize using onInit to elect master
> ---------------------------------------------------------------------
>
>                 Key: CAMEL-5719
>                 URL: https://issues.apache.org/jira/browse/CAMEL-5719
>             Project: Camel
>          Issue Type: Improvement
>          Components: camel-zookeeper
>    Affects Versions: 2.10.0
>            Reporter: Claus Ibsen
>             Fix For: Future
>
>         Attachments: zookeeper-election.patch
>
>
> See nabble
> http://camel.465427.n5.nabble.com/Single-instance-of-running-route-in-the-cluster-tp5720846.html



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message