storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rakesh Nair <raknai...@gmail.com>
Subject Too many failed tuples
Date Fri, 03 Jan 2014 02:27:31 GMT
I am working on a bolt (extends BaseRichBolt) which writes the tuples into
an hdfs file. I am batching the tuples in the bolt and once the batch size
reaches a certain value i write them to the hdfs file and at the same time
send out ack's for those tuples.

To prevent timeouts i have set the topology configs
TOPOLOGY_ENABLE_MESSAGE_TIMEOUTS to true and TOPOLOGY_MESSAGE_TIMEOUT_SECS
to a large enough value.

When i check the status of the topology in the UI, i see that bolt is
showing way more acked tuples compared to the spout (its 3x more) and then
for the spout configs i see tons of failed tuples.
I dont see any failures or exception in the log files so i guess the
program is working just fine. The failures in the spout could have been
because of tuple timeout (caused by batching and hdfs write latency etc),
but then since i have set the config TOPOLOGY_MESSAGE_TIMEOUT_SECS to a
high enough value, i dont expect that to happen.

Is there something obvious that i am missing or doing wrong ?

-- 
Regards
Rakesh Nair

Mime
View raw message