storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stig Rohde Døssing <s...@apache.org>
Subject Re: Emitting tuples anchored to already-acked tuples
Date Fri, 03 Nov 2017 15:17:21 GMT
Hi,

It depends a bit on which order the acker bolt sees the messages.

When bolt B acks the tuple, it sends a value to the acker bolt that
essentially communicates "I'm done with tuple 123, and I didn't emit any
new tuples anchored to it". Please read the section "How does Storm
implement reliability in an efficient way" at
https://storm.apache.org/releases/2.0.0-SNAPSHOT/Guaranteeing-message-processing.html
for details.

If the acker receives the ack from bolt B first, it will forward the ack to
the spout and forget about the tuple tree. Any other messages for that
tuple tree will effectively be ignored.

If the acker receives the ack from bolt C first (let's say the ack from B
was delayed for some reason), the tuple ack val won't match up to what the
acker expects. The acker will essentially be receiving acks for tuples it
doesn't know about. Since the acking mechanism is based on acking tuple ids
with themselves to arrive at a value of 0 when the full tree is acked, this
will prevent the tree from being acked. The tuple on the spout will time
out and fail. In case the message from C is a fail, it will act like you
expect and fail the tuple immediately on the spout.

The bottom line is to avoid anchoring new tuples to acked tuples :)

You might find
https://github.com/apache/storm/blob/6207d320c3f79185f5e8e19458d73fbf1aa10f72/storm-client/src/jvm/org/apache/storm/daemon/Acker.java#L80
and
https://github.com/apache/storm/blob/6207d320c3f79185f5e8e19458d73fbf1aa10f72/storm-client/src/jvm/org/apache/storm/executor/bolt/BoltOutputCollectorImpl.java#L79
interesting.

2017-11-03 14:27 GMT+01:00 Lawrence Craft <lawrence.craft@gmail.com>:

> Hi all,
>
> Out of curiosity, what happens if a bolt emits tuples anchored to spout
> tuples that have already been acked?
>
> As an example: Spout A emits tuple with ID 123. Bolt B receives and
> immediately acks tuple 123, but keeps it around in memory. At a later
> point, something triggers Bolt B to emit a tuple anchored to the original
> spout tuple 123. This new tuple is received by Bolt C.
>
> What happens if Bolt C fails this tuple? Or acks it? Will the spout
> receive a duplicate ack (or an ack followed by a fail)?
>
> Thanks,
>
> Lawrence
>

Mime
View raw message