hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sonal Goyal <sonalgoy...@gmail.com>
Subject Re: HBase starting problem
Date Fri, 23 Mar 2012 11:42:25 GMT
Your logs still point to a permissions issue on /hbase, so I would start at
resolving that first thing..

org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:76)
   at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:1602)
*Caused by: org.apache.zookeeper.KeeperException$AuthFailedException:
KeeperErrorCode = AuthFailed for /hbase
   at org.apache.zookeeper.**KeeperException.creat*e(
KeeperException.java:123)
   at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)


Best Regards,
Sonal
Crux: Reporting for HBase <https://github.com/sonalgoyal/crux>
Nube Technologies <http://www.nubetech.co>

<http://in.linkedin.com/in/sonalgoyal>





On Thu, Mar 22, 2012 at 6:56 PM, Stijn Diependaele <sdiepend@gmail.com>wrote:

> Thanks for your reply,
>
> Currently I have no special permissions set for dfs, I think it's all
> default.
> It's the root that's running the dfs, same for zookeeper it's also the
> root.
> But for some reason the log from the hbase master is stored in Stijn, which
> is a normal user, I'm not sure if this has anything to do with the problem.
> Normally it should be root that runs it so it should be logged into the
> hbase-root-master... file.
>
> logs/hbase-Stijn-master-HADOOP.domain.be.log
>
> Now I've removed the hbase.keytab and krb5.keytab and I made new ones. Now
> I'm getting a different error:
> And it seems that some problem could be the resolving of the hostname, but
> normally that's configured well.
>
> .....
> .....
> 2012-03-22 14:19:28,166 INFO org.apache.zookeeper.ClientCnxn: Session
> establishment complete on server HDP06.domain.be/10.11.12.206:2222,
> sessionid = 0x1363a8fc39d0001, negotiated timeout = 180000
> 2012-03-22 14:19:28,194 ERROR
> org.apache.zookeeper.client.ZooKeeperSaslClient: An error:
> (java.security.PrivilegedActionException:
> javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Server not
> found in Kerberos database (7) - UNKNOWN_SERVER)]) occurred when evaluating
> Zookeeper Quorum Member's  received SASL token. This may be caused by
> Java's being unable to resolve the Zookeeper Quorum Member's hostname
> correctly. You may want to try to adding
> '-Dsun.net.spi.nameservice.provider.1=dns,sun' to your client's JVMFLAGS
> environment. Zookeeper Client will go to AUTH_FAILED state.
> 2012-03-22 14:19:28,194 ERROR org.apache.zookeeper.ClientCnxn: SASL
> authentication with Zookeeper Quorum member failed:
> javax.security.sasl.SaslException: An error:
> (java.security.PrivilegedActionException:
> javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Server not
> found in Kerberos database (7) - UNKNOWN_SERVER)]) occurred when evaluating
> Zookeeper Quorum Member's  received SASL token. This may be caused by
> Java's being unable to resolve the Zookeeper Quorum Member's hostname
> correctly. You may want to try to adding
> '-Dsun.net.spi.nameservice.provider.1=dns,sun' to your client's JVMFLAGS
> environment. Zookeeper Client will go to AUTH_FAILED state.
> 2012-03-22 14:19:28,229 ERROR
> org.apache.hadoop.hbase.master.HMasterCommandLine: Failed to start master
> java.lang.RuntimeException: Failed construction of Master: class
> org.apache.hadoop.hbase.master.HMaster
>    at
> org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:1588)
>    at
>
> org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:144)
>    at
>
> org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:103)
>    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>    at
>
> org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:76)
>    at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:1602)
> Caused by: org.apache.zookeeper.KeeperException$AuthFailedException:
> KeeperErrorCode = AuthFailed for /hbase
>    at org.apache.zookeeper.KeeperException.create(KeeperException.java:123)
>    at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
>    at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1003)
>    at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1031)
>    at
>
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:189)
>    at
>
> org.apache.hadoop.hbase.zookeeper.ZKUtil.createAndFailSilent(ZKUtil.java:892)
>    at
>
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.createBaseZNodes(ZooKeeperWatcher.java:159)
>    at
>
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.<init>(ZooKeeperWatcher.java:152)
>    at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:264)
>    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
>    at
>
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
>    at
>
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>    at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
>    at
> org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:1583)
>    ... 5 more
>
>
> 2012/3/21 Sonal Goyal <sonalgoyal4@gmail.com>
>
> > Hi Stijn,
> >
> > You seem to be hitting permission issues with /hbase
> >
> > Caused by: org.apache.zookeeper.KeeperException$InvalidACLException:
> > KeeperErrorCode = InvalidACL for /hbase
> >   at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:121)
> >   at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> >   at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:778)
> >
> > What are your dfs user permission settings like? Which user are you
> running
> > hbase as?
> >
> > Best Regards,
> > Sonal
> > Crux: Reporting for HBase <https://github.com/sonalgoyal/crux>
> > Nube Technologies <http://www.nubetech.co>
> >
> > <http://in.linkedin.com/in/sonalgoyal>
> >
> >
> >
> >
> >
> > On Wed, Mar 21, 2012 at 3:49 PM, Stijn Diependaele <sdiepend@gmail.com
> > >wrote:
> >
> > > I'm trying to setup HBase on top of my HDFS filesystem.
> > > I currently have 6 servers but only 3 are configured to run zookeeper
> and
> > > HBase, for testing purposes and the idea of adding the other servers
> when
> > > this works.
> > > I'm following this guide
> > http://hbase.apache.org/book/configuration.htmlto
> > > set it up.
> > >
> > > I can paste the entire log in pastebin if you want.
> > > HBase-site.xml : http://pastebin.com/Gb8UFSHj
> > >
> > > Errors in the log files:
> > >
> > > Zookeeper:
> > > ----------
> > >
> > > logs/hbase-root-zookeeper-HADOOP.domain.be.log (master)
> > > ....
> > > ....
> > > 2012-03-21 09:58:39,576 INFO
> org.apache.zookeeper.server.ZooKeeperServer:
> > > Established session 0x36347aae9a0000 with negotiated timeout 180000 for
> > > client /10.11.12.206:43325
> > > 2012-03-21 09:58:39,611 WARN org.apache.zookeeper.server.NIOServerCnxn:
> > > caught end of stream exception
> > > EndOfStreamException: Unable to read additional data from client
> > sessionid
> > > 0x36347aae9a0000, likely client has closed socket
> > >    at
> > > org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
> > >    at
> > >
> > >
> >
> org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:224)
> > >    at java.lang.Thread.run(Thread.java:722)
> > > 2012-03-21 09:58:39,617 INFO org.apache.zookeeper.server.NIOServerCnxn:
> > > Closed socket connection for client /10.11.12.206:43325 which had
> > > sessionid
> > > 0x36347aae9a0000
> > > ....
> > > ....
> > >
> > > on a znode:
> > > logs/hbase-root-zookeeper-HDP08.domain.be.log
> > > .....
> > > .....
> > > 2012-03-21 09:58:31,290 INFO
> > > org.apache.zookeeper.server.quorum.QuorumPeerConfig: Defaulting to
> > majority
> > > quorums
> > > 2012-03-21 09:58:31,433 DEBUG org.apache.hadoop.hbase.util.VersionInfo:
> > > preRegister called.
> Server=com.sun.jmx.mbeanserver.JmxMBeanServer@4a1908
> > ,
> > > name=log4j:logger=org.apache.hadoop.hbase.util.VersionInfo
> > > 2012-03-21 09:58:31,434 DEBUG
> org.apache.hadoop.hbase.zookeeper.ZKConfig:
> > > preRegister called.
> Server=com.sun.jmx.mbeanserver.JmxMBeanServer@4a1908
> > ,
> > > name=log4j:logger=org.apache.hadoop.hbase.zookeeper.ZKConfig
> > > 2012-03-21 09:58:31,434 DEBUG
> org.apache.hadoop.hbase.HBaseConfiguration:
> > > preRegister called.
> Server=com.sun.jmx.mbeanserver.JmxMBeanServer@4a1908
> > ,
> > > name=log4j:logger=org.apache.hadoop.hbase.HBaseConfiguration
> > > 2012-03-21 09:58:31,435 DEBUG org.apache.hadoop.hbase: preRegister
> > called.
> > > Server=com.sun.jmx.mbeanserver.JmxMBeanServer@4a1908,
> > > name=log4j:logger=org.apache.hadoop.hbase
> > > 2012-03-21 09:58:31,435 INFO
> > > org.apache.zookeeper.server.quorum.QuorumPeerMain: Starting quorum peer
> > > 2012-03-21 09:58:31,836 INFO org.apache.zookeeper.Login: successfully
> > > logged in.
> > > 2012-03-21 09:58:31,842 INFO org.apache.zookeeper.Login: TGT refresh
> > thread
> > > started.
> > > .....
> > > .....
> > > a bit further down the log file:
> > > ....
> > > ....
> > > 2012-03-21 09:58:32,425 INFO
> > > org.apache.zookeeper.server.quorum.LearnerHandler: Sending snapshot
> last
> > > zxid of peer is 0x1800000012  zxid of leader is 0x1900000000sent zxid
> of
> > db
> > > as 0x1800000012
> > > 2012-03-21 09:58:32,466 WARN org.apache.zookeeper.server.quorum.Leader:
> > > Commiting zxid 0x1900000000 from /10.11.12.208:2888 not first!
> > > 2012-03-21 09:58:32,466 WARN org.apache.zookeeper.server.quorum.Leader:
> > > First is 0
> > > 2012-03-21 09:58:32,466 INFO org.apache.zookeeper.server.quorum.Leader:
> > > Have quorum of supporters; starting up and setting last processed zxid:
> > > 107374182400
> > > 2012-03-21 09:58:36,505 INFO
> > > org.apache.zookeeper.server.quorum.QuorumCnxManager: Received
> connection
> > > request /10.11.12.200:46015
> > > ....
> > > ....
> > >
> > > HBase:
> > > -----
> > > master:
> > > ....
> > > ....
> > > 2012-03-21 09:58:39,162 WARN org.apache.zookeeper.ClientCnxn: Zookeeper
> > > client cannot authenticate using the Client section of the supplied
> > > configuration file: '/home/Stijn/hbase-0.92.0/conf/jaas.conf'. Will
> > > continue connection to Zookeeper server without SASL authentication, if
> > > Zookeeper server allows it.
> > > ....
> > > ....
> > > 2012-03-21 09:58:39,313 INFO org.apache.zookeeper.ClientCnxn: Session
> > > establishment complete on server HDP06.domain.be/10.11.12.206:2222,
> > > sessionid = 0x136347a9db40000, negotiated timeout = 180000
> > > 2012-03-21 09:58:39,454 ERROR
> > > org.apache.hadoop.hbase.master.HMasterCommandLine: Failed to start
> master
> > > java.lang.RuntimeException: Failed construction of Master: class
> > > org.apache.hadoop.hbase.master.HMaster
> > >    at
> > >
> org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:1588)
> > >    at
> > >
> > >
> >
> org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:144)
> > >    at
> > >
> > >
> >
> org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:103)
> > >    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> > >    at
> > >
> > >
> >
> org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:76)
> > >    at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:1602)
> > > Caused by: org.apache.zookeeper.KeeperException$InvalidACLException:
> > > KeeperErrorCode = InvalidACL for /hbase
> > >    at
> > org.apache.zookeeper.KeeperException.create(KeeperException.java:121)
> > >    at
> > org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> > >    at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:778)
> > >    .....
> > >    .....
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message