bookkeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jiannan Wang <jian...@yahoo-inc.com>
Subject Re: [Discussion] [Hedwig] Add queue semantic support for Hedwig
Date Sat, 23 Feb 2013 08:55:50 GMT
Thanks Yannick for your reply, I go over mailing list and find you had mentioned the problem
before.

The two points your talk is interesting:
   - I think message "life time" is another topic, our problem is on the consume pointer not
on message live time (even message is live but the sub is after the message, subscriber still
cannot receive message). But it's awesome if Hedwig could support this feature.
   - Message "minimum consummation number", I guess you mean "minimum consummation number
by different subscribers" which changes current message garbage collection strategy. It can
resolve the problem but I don't know if it's worth to add this feature, is there any other
usage scenario?

- Jiannan


From: Yannick Legros <yannick.legros59@gmail.com<mailto:yannick.legros59@gmail.com>>
Reply-To: "bookkeeper-user@zookeeper.apache.org<mailto:bookkeeper-user@zookeeper.apache.org>"
<bookkeeper-user@zookeeper.apache.org<mailto:bookkeeper-user@zookeeper.apache.org>>
Date: Thursday, February 21, 2013 4:25 PM
To: "bookkeeper-user@zookeeper.apache.org<mailto:bookkeeper-user@zookeeper.apache.org>"
<bookkeeper-user@zookeeper.apache.org<mailto:bookkeeper-user@zookeeper.apache.org>>
Subject: Re: [Discussion] [Hedwig] Add queue semantic support for Hedwig

Hi Jiannan

I already face to this problem with Hedwig and i told this on this mailing list few weeks
ago.
And yes the main problem is too coordinate publisher and subscriber to make sure the first
SUB is handled before the first PUB.

just to add my contribution to this reflection :

1.  I think about some "life time" or "minumum consumation number" parameters for messages.

Regards,
Yannick.




2013/2/21 Jiannan Wang <jiannan@yahoo-inc.com<mailto:jiannan@yahoo-inc.com>>
Hi guys,
   Under current Hedwig semantic, a subscriber cannot aware of messages published before he
subscribes the topic. So in following example, subscriber A can only receives messages after
seqId 2.
---------------------------------
Topic T: msg1 msg2 msg3 msg4 ...
                     | <- subscriber A subscribe the topic
---------------------------------

   This semantic is very reasonable, but Hedwig client needs to handle this corner case: a
new topic is just to be created, and as topic is lazily created by the first request (generally
it's PUB or SUB), so the client side must coordinate between publisher and subscriber to make
sure the first SUB is handled before the first PUB at this very beginning status (consider
subscriber may have very bad network connection which causes SUB failed and user does not
want to miss any messages). In summary, it requires special works if there is a subscriber
would like to receive all the messages since topic is created, and I think this requirement
is very general.

   Handle this problem in client side is a choice, but I think maybe we can simply resolve
it in server side if Hedwig can support queue semantic (so that we can also extend Hedwig
JMS provider to support JMS queue in BOOKKEEPER-312). And as I known, the major concern on
queue semantic is how long to keep the messages, however:
   1. It is user's responsibility to know about the feature and impact of queue semantic.
   2. On the other hand, we can add a parameter to limit the queue length.

   In a word, here are the two problem I would like to discuss:
   1. How to gracefully resolve the above issue in server side under current semantic.
   2. Whether or not to introduce queue semantic into Hedwig.

Thanks,
Jiannan


Mime
View raw message