hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ravi Prakash <ravihad...@gmail.com>
Subject Re: New cluster help
Date Thu, 14 Jul 2016 21:57:10 GMT
Hi Tombin!

Is this the first cluster you're ever setting up? Are you able to run an
"hdfs dfs -ls /" successfully? How about putting files into HDFS? I'd take
it one step at a time if I were you. i.e.

1. Set up a simple HDFS cluster (without SSL)
2. Turn on SSL
3. Then try to run HBase.

Is step 1 working for you?

Ravi

On Thu, Jul 14, 2016 at 12:59 PM, tombin <tombin.ansi@gmail.com> wrote:

> I am setting up a new hadoop cluster for the first time.  My setup
> currently looks as follows:
>
> hdfs cluster:
> 1 namenode
> 2 datanodes
>
> hbase:
> 1 hbase node
>
> zookeeper cluster:
> 3 zookeeper nodes
>
> I have enabled ssl on the hdfs cluster.  When trying to connect from base
> i see the following error:
>
> 2016-07-14 19:38:58,333 WARN  [Thread-73] hdfs.DFSClient: DataStreamer
> Exception
>
> java.lang.NullPointerException
>
> at
> org.apache.hadoop.hdfs.protocol.datatransfer.DataTransferEncryptor.getEncryptedStreams(DataTransferEncryptor.java:191)
>
> at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:1335)
>
> at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1281)
>
> at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:526)
>
> 2016-07-14 19:39:04,341 INFO  [hb01:16000.activeMasterManager]
> hdfs.DFSClient: Could not complete /hbase/.tmp/hbase.version retrying...
>
>
>
> this will repeat several time and then i'll throw the following exception:
>
>
> 2016-07-14 19:39:58,772 FATAL [hb01:16000.activeMasterManager]
> master.HMaster: Unhandled exception. Starting shutdown.
>
> java.io.IOException: Unable to close file because the last block does not
> have enough number of replicas.
>
> at
> org.apache.hadoop.hdfs.DFSOutputStream.completeFile(DFSOutputStream.java:2151)
>
> at org.apache.hadoop.hdfs.DFSOutputStream.close(DFSOutputStream.java:2119)
>
> at
> org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:72)
>
> at
> org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:106)
>
> at org.apache.hadoop.hbase.util.FSUtils.setVersion(FSUtils.java:730)
>
> at org.apache.hadoop.hbase.util.FSUtils.setVersion(FSUtils.java:705)
>
> at org.apache.hadoop.hbase.util.FSUtils.checkVersion(FSUtils.java:662)
>
> at
> org.apache.hadoop.hbase.master.MasterFileSystem.checkRootDir(MasterFileSystem.java:462)
>
> at
> org.apache.hadoop.hbase.master.MasterFileSystem.createInitialFileSystemLayout(MasterFileSystem.java:153)
>
> at
> org.apache.hadoop.hbase.master.MasterFileSystem.<init>(MasterFileSystem.java:128)
>
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:652)
>
> at org.apache.hadoop.hbase.master.HMaster.access$600(HMaster.java:185)
>
> at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1750)
>
> at java.lang.Thread.run(Thread.java:745)
>
>
> hbase shuts down at this point.
>
>
> on the datanode i side i see the following in the logs that looks like it
> may be related:
>
>
> 2016-07-14 19:38:23,132 ERROR
> org.apache.hadoop.hdfs.server.datanode.DataNode: hd03.domain.com:50010:DataXceiver
> error processing unknown operation  src: /10.0.0.10:34893 dst: /
> 10.0.1.10:50010
>
> java.io.EOFException
>
>         at java.io.DataInputStream.readInt(DataInputStream.java:392)
>
>         at
> org.apache.hadoop.hdfs.protocol.datatransfer.sasl.SaslDataTransferServer.doSaslHandshake(SaslDataTransferServer.java:358)
>
>         at
> org.apache.hadoop.hdfs.protocol.datatransfer.sasl.SaslDataTransferServer.getEncryptedStreams(SaslDataTransferServer.java:178)
>
>         at
> org.apache.hadoop.hdfs.protocol.datatransfer.sasl.SaslDataTransferServer.receive(SaslDataTransferServer.java:110)
>
>         at
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:193)
>
>         at java.lang.Thread.run(Thread.java:745)
>
> 2016-07-14 19:39:33,575 ERROR
> org.apache.hadoop.hdfs.server.datanode.DataNode: hd03.domain.com:50010:DataXceiver
> error processing unknown operation  src: /10.0.0.10:34898 dst: /
> 10.0.1.10:50010
>
> java.io.EOFException
>
>         at java.io.DataInputStream.readInt(DataInputStream.java:392)
>
>         at
> org.apache.hadoop.hdfs.protocol.datatransfer.sasl.SaslDataTransferServer.doSaslHandshake(SaslDataTransferServer.java:358)
>
>         at
> org.apache.hadoop.hdfs.protocol.datatransfer.sasl.SaslDataTransferServer.getEncryptedStreams(SaslDataTransferServer.java:178)
>
> Is this in relation to my ssl configuration ?
> I'm confused about whats going on here.  Thank you in advance for any help.
>

Mime
View raw message