storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Raja.Aravapalli <>
Subject hdfsbolt in secure cluster failing with
Date Tue, 16 Feb 2016 07:37:36 GMT


I have a hdfs bolt in my storm topology, it is running perfectly fine for some time, but suddently
it is failing with SASL security exceptions!!

I have followed the details outlined [Working with Secure HDFS - Using key tabs on all worker
hosts] at link as we are on Kerberos
secured cluster.

Shared below is the code i am using to configure hdfs bolt !

builder.setBolt("hdfs-bolt-1", HDFSBolt, 1)

builder.setBolt("hdfs-bolt-2", invalidRecsHDFSBolt, 1)

Below is the exception: IOException Failed on local exception: GSS initiate failed [Caused by GSSException: No valid credentials
provided (Mechanism level: Failed to find any Kerberos tgt)]; Host Details : local host is:
"xxxxxxxxxxxx"; destination host is: "xxxxxxxx":8020;

        at org.apache.hadoop.hdfs.DFSOutputStream.flushOrSync( ~[stormjar.jar:na]

        at org.apache.hadoop.hdfs.DFSOutputStream.hsync( ~[stormjar.jar:na]

        at org.apache.hadoop.hdfs.client.HdfsDataOutputStream.hsync(

        at org.apache.storm.hdfs.bolt.HdfsBolt.execute( ~[stormjar.jar:na]

        at backtype.storm.daemon.executor$fn__5265$tuple_action_fn__5267.invoke(executor.clj:659)

        at backtype.storm.daemon.executor$mk_task_receiver$fn__5188.invoke(executor.clj:415)

        at backtype.storm.disruptor$clojure_handler$reify__1064.onEvent(disruptor.clj:58)

        at backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(

        at backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(

        at backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:80)

        at backtype.storm.daemon.executor$fn__5265$fn__5278$fn__5329.invoke(executor.clj:794)

        at backtype.storm.util$async_loop$fn__551.invoke(util.clj:465) [storm-core-]

        at [clojure-1.5.1.jar:na]

        at [na:1.7.0_45]

Some where i read, storm hdfs bolt fails to authenticate, when multiple tasks/executors used!!

Can someone pls help me fix the issue... Please share your thoughts.... to fix the issue.

Raja Aravapalli

View raw message