hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan Gray <jg...@fb.com>
Subject RE: I give up, help please
Date Wed, 22 Dec 2010 01:46:15 GMT
>> Caused by: java.io.EOFException
>>          at
>> java.io.DataInputStream.readUnsignedShort(DataInputStream.java:323)
>>          at java.io.DataInputStream.readUTF(DataInputStream.java:572)
>>          at
>> org.apache.hadoop.hbase.util.FSUtils.getVersion(FSUtils.java:151)
>>          at
>> org.apache.hadoop.hbase.util.FSUtils.checkVersion(FSUtils.java:170)
>>          at
>> org.apache.hadoop.hbase.master.HMaster.checkRootDir(HMaster.java:304)
>>          at
>> org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:221)
>>          ... 15 more

This is reading a file in the hbase root directory.  This shouldn't happen if it doesn't exist.
 I could be wrong because I haven't looked at the code yet.

Also, I'm not sure why a master process would appear after running stop.  Do you have multiple
master processes running somehow?  If you ps auxww | grep java do u see more than you see
with jps?

> -----Original Message-----
> From: Pete Haidinyak [mailto:javamann@cox.net]
> Sent: Tuesday, December 21, 2010 5:41 PM
> To: user@hbase.apache.org
> Subject: Re: I give up, help please
> 
> Tried that, same thing. Quick question, when I do a 'stop-hbase.sh'
> zookeeper dies but then an HMaster process appears?
> 
> Thanks
> 
> -Pete
> 
> 
> On Tue, 21 Dec 2010 17:24:36 -0800, Jonathan Gray <jgray@fb.com> wrote:
> 
> > You have existing data?  Try clearing out your hbase directory in hdfs.
> > Looks like some weird problem reading the hbase.version file out of HDFS.
> >
> >> -----Original Message-----
> >> From: Pete Haidinyak [mailto:javamann@cox.net]
> >> Sent: Tuesday, December 21, 2010 3:32 PM
> >> To: HBase Group
> >> Subject: I give up, help please
> >>
> >> After 11 hours I give up. I am trying to get a 5 node system up on my
> >> ESXi server.
> >> OS CentOS 5.5 x64
> >> Hadoop Version 0.20.2+737
> >> HBase Version 0.87.20100924+28
> >>
> >> Name Node - Zoo Keeper - HBase Master on one machine
> >> 4 Region Server
> >>
> >> Hadoop starts up on every node without any problems. When I try to
> >> start HBase on the Master server I get zookeeper coming up but no
> >> HBase Master.
> >> When I do a hbase-stop.sh the zookeeper goes away and then the
> >> HMaster appears in a 'JPS' command. Here is the log file
> >>
> >>
> >> Tue Dec 21 18:17:35 EST 2010 Starting master on hadoop-name-node
> >> ulimit -n
> >> 1024
> >> 2010-12-21 18:17:36,380 INFO
> >> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Reconnecting
> to
> >> zookeeper
> >> 2010-12-21 18:17:36,462 INFO org.apache.zookeeper.ZooKeeper: Client
> >> environment:zookeeper.version=3.3.1-942149, built on 05/07/2010 17:14
> >> GMT
> >> 2010-12-21 18:17:36,462 INFO org.apache.zookeeper.ZooKeeper: Client
> >> environment:host.name=hadoop-name-node
> >> 2010-12-21 18:17:36,462 INFO org.apache.zookeeper.ZooKeeper: Client
> >> environment:java.version=1.6.0_23
> >> 2010-12-21 18:17:36,462 INFO org.apache.zookeeper.ZooKeeper: Client
> >> environment:java.vendor=Sun Microsystems Inc.
> >> 2010-12-21 18:17:36,462 INFO org.apache.zookeeper.ZooKeeper: Client
> >> environment:java.home=/usr/java/jdk1.6.0_23/jre
> >> 2010-12-21 18:17:36,462 INFO org.apache.zookeeper.ZooKeeper: Client
> >> environment:java.class.path=/usr/local/hbase/conf:/usr/java/default/l
> >> ib/to
> >> ols.jar::/usr/local/hbase:/usr/local/hbase/hbase-0.89.2010
> >> 0924+28.jar:/usr/local/hbase/hbase-0.89.20100924+28-tests.jar:/usr/lo
> >> 0924+cal
> >> 0924+/hbase/lib/activation-1.1.jar:/usr/local/hbase/lib/ant-1.6.5.jar
> >> 0924+:/u
> >> 0924+sr/local/hbase/lib/ant-1.7.1.jar:/usr/local/hbase/lib/ant-lau
> >> ncher-1.7.1.jar:/usr/local/hbase/lib/asm-3.1.jar:/usr/local/hbase/lib
> >> /avro-
> >> 1.3.2.jar:/usr/local/hbase/lib/commons-cli-
> >> 1.2.jar:/usr/local/hbase/lib/commons-codec-
> >> 1.2.jar:/usr/local/hbase/lib/commons-el
> >> -1.0.jar:/usr/local/hbase/lib/commons-httpclient-
> >> 3.1.jar:/usr/local/hbase/lib/commons-lang-
> >> 2.5.jar:/usr/local/hbase/lib/commons-logging-
> >> 1.1.1.jar:/usr/local/hbase/lib/commons-net-1.4.1.jar:/usr/local/
> >> hbase/lib/core-3.1.1.jar:/usr/local/hbase/lib/guava-
> >> r05.jar:/usr/local/hbase/lib/hadoop-core-0.20.2-
> >> 737.jar:/usr/local/hbase/lib/hsqldb-1.8.0.10.jar:/usr/local/hbase/lib
> >> /jackson-
> >> core-asl-1.5.2.jar:/us
> >> r/local/hbase/lib/jackson-mapper-asl-1.5.2.jar:/usr/local/hbase/lib/j
> >> asper-
> >> compiler-5.5.23.jar:/usr/local/hbase/lib/jasper-runtime-
> >> 5.5.23.jar:/usr/local/hbase/lib/jaxb-api-2.1.jar:/usr/local/hbase/lib
> >> /jaxb-impl-2.1.12.jar:/usr/local/hbase/lib/jersey-core-
> >> 1.1.5.1.jar:/usr/local/hbase/lib/jersey-json-
> >> 1.1.5.1.jar:/usr/local/hbase/lib/jersey-server-
> >> 1.1.5.1.jar:/usr/local/hbase/lib/jets3t-0.7.1.jar:/us
> >> r/local/hbase/lib/jettison-1.1.jar:/usr/local/hbase/lib/jetty-
> >> 6.1.24.jar:/usr/local/hbase/lib/jetty-util-6.1.24.jar:/usr/local/hbas
> >> e/lib/jline-
> >> 0.9.94.jar:/usr/local/hbase/lib/jruby-complete-1.4.0.jar:
> >> /usr/local/hbase/lib/jsp-2.1-6.1.14.jar:/usr/local/hbase/lib/jsp-api-
> >> 2.0.jar:/usr/local/hbase/lib/jsp-api-2.1-6.1.14.jar:/usr/local/hbase/
> >> lib/jsr311- api-1.1.1.jar:/usr/local/hbase/lib/kfs-0.3.jar:/usr
> >> /local/hbase/lib/log4j-1.2.15.jar:/usr/local/hbase/lib/oro-
> >> 2.0.8.jar:/usr/local/hbase/lib/paranamer-
> >> 2.2.jar:/usr/local/hbase/lib/paranamer-ant-
> >> 2.2.jar:/usr/local/hbase/lib/paranamer-generator-2.2.jar:
> >> /usr/local/hbase/lib/protobuf-java-2.3.0.jar:/usr/local/hbase/lib/qdo
> >> x-
> >> 1.10.1.jar:/usr/local/hbase/lib/servlet-api-
> >> 2.4.jar:/usr/local/hbase/lib/servlet-api-2.5-
> >> 20081211.jar:/usr/local/hbase/lib/servle
> >> t-api-2.5-6.1.14.jar:/usr/local/hbase/lib/slf4j-api-
> >> 1.5.8.jar:/usr/local/hbase/lib/slf4j-log4j12-1.5.8.jar:/usr/local/hba
> >> se/lib/stax-
> >> api-1.0.1.jar:/usr/local/hbase/lib/stax-api-1.0-2.jar:/usr/local/hb
> >> ase/lib/thrift-0.2.0.jar:/usr/local/hbase/lib/xmlenc-
> >> 0.52.jar:/usr/local/hbase/lib/zookeeper-
> >> 3.3.1.jar:/etc/zookeeper:/etc/hadoop-0.20/conf
> >> 2010-12-21 18:17:36,462 INFO org.apache.zookeeper.ZooKeeper: Client
> >> environment:java.library.path=/usr/java/jdk1.6.0_23/jre/lib/amd64/ser
> >> ver:/ usr/java/jdk1.6.0_23/jre/lib/amd64:/usr/java/jdk1.6.0_23/j
> >> re/../lib/amd64:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/
> >> usr/lib
> >> 2010-12-21 18:17:36,462 INFO org.apache.zookeeper.ZooKeeper: Client
> >> environment:java.io.tmpdir=/tmp
> >> 2010-12-21 18:17:36,462 INFO org.apache.zookeeper.ZooKeeper: Client
> >> environment:java.compiler=<NA>
> >> 2010-12-21 18:17:36,462 INFO org.apache.zookeeper.ZooKeeper: Client
> >> environment:os.name=Linux
> >> 2010-12-21 18:17:36,462 INFO org.apache.zookeeper.ZooKeeper: Client
> >> environment:os.arch=amd64
> >> 2010-12-21 18:17:36,462 INFO org.apache.zookeeper.ZooKeeper: Client
> >> environment:os.version=2.6.18-194.3.1.el5
> >> 2010-12-21 18:17:36,463 INFO org.apache.zookeeper.ZooKeeper: Client
> >> environment:user.name=hadoop
> >> 2010-12-21 18:17:36,463 INFO org.apache.zookeeper.ZooKeeper: Client
> >> environment:user.home=/home/hadoop
> >> 2010-12-21 18:17:36,463 INFO org.apache.zookeeper.ZooKeeper: Client
> >> environment:user.dir=/usr/local/hbase/logs
> >> 2010-12-21 18:17:36,464 INFO org.apache.zookeeper.ZooKeeper:
> >> Initiating client connection, connectString=hadoop-name-node:2222
> >> sessionTimeout=60000
> >> watcher=org.apache.hadoop.hbase.zookeeper.ZooKeeperW
> >> rapper@1bbb60c3
> >> 2010-12-21 18:17:36,502 INFO org.apache.zookeeper.ClientCnxn: Opening
> >> socket connection to server hadoop-name-node/192.168.1.210:2222
> >> 2010-12-21 18:17:36,508 DEBUG
> >> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper:
> >> <hadoop-name-
> >> node:/hbase,org.apache.hadoop.hbase.master.HMaster>Connected
> >> to zookeeper again
> >> 2010-12-21 18:17:36,516 INFO org.apache.zookeeper.ClientCnxn: Socket
> >> connection established to hadoop-name-node/192.168.1.210:2222,
> >> initiating
> >> session
> >> 2010-12-21 18:17:36,542 INFO org.apache.zookeeper.ClientCnxn: Session
> >> establishment complete on server hadoop-name-
> node/192.168.1.210:2222,
> >> sessionid = 0x12d0b3937d20000, negotiated timeout = 60000
> >> 2010-12-21 18:17:36,569 INFO org.apache.hadoop.ipc.HBaseServer:
> >> Starting SocketReader
> >> 2010-12-21 18:17:36,570 INFO org.apache.hadoop.ipc.HBaseServer:
> >> Starting SocketReader
> >> 2010-12-21 18:17:36,570 INFO org.apache.hadoop.ipc.HBaseServer:
> >> Starting SocketReader
> >> 2010-12-21 18:17:36,571 INFO org.apache.hadoop.ipc.HBaseServer:
> >> Starting SocketReader
> >> 2010-12-21 18:17:36,571 INFO org.apache.hadoop.ipc.HBaseServer:
> >> Starting SocketReader
> >> 2010-12-21 18:17:36,572 INFO org.apache.hadoop.ipc.HBaseServer:
> >> Starting SocketReader
> >> 2010-12-21 18:17:36,574 INFO org.apache.hadoop.ipc.HBaseServer:
> >> Starting SocketReader
> >> 2010-12-21 18:17:36,576 INFO org.apache.hadoop.ipc.HBaseServer:
> >> Starting SocketReader
> >> 2010-12-21 18:17:36,578 INFO
> >> org.apache.hadoop.hbase.ipc.HBaseRpcMetrics:
> >> Initializing RPC Metrics with hostName=HMaster, port=60000
> >> 2010-12-21 18:17:36,583 INFO org.apache.hadoop.ipc.HBaseServer:
> >> Starting SocketReader
> >> 2010-12-21 18:17:36,585 INFO org.apache.hadoop.ipc.HBaseServer:
> >> Starting SocketReader
> >> 2010-12-21 18:17:36,880 ERROR
> >> org.apache.hadoop.hbase.master.HMasterCommandLine: Failed to start
> >> master
> >> java.lang.RuntimeException: Failed construction of Master: class
> >> org.apache.hadoop.hbase.master.HMasternull
> >>          at
> >>
> org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:1
> >> 243)
> >>          at
> >>
> org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMas
> >> terCommandLine.java:143)
> >>          at
> >>
> org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCom
> >> mandLine.java:102)
> >>          at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> >>          at
> >>
> org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerComman
> >> dLine.java:76)
> >>          at
> >> org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:1286)
> >> Caused by: java.lang.reflect.UndeclaredThrowableException: Unknown
> >> exception in doAs
> >>          at
> >>
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInform
> >> ation.java:1075)
> >>          at
> >>
> org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:1
> >> 236)
> >>          ... 5 more
> >> Caused by: java.security.PrivilegedActionException:
> >> java.lang.reflect.InvocationTargetException
> >>          at java.security.AccessController.doPrivileged(Native Method)
> >>          at javax.security.auth.Subject.doAs(Subject.java:396)
> >>          at
> >>
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInform
> >> ation.java:1063)
> >>          ... 6 more
> >> Caused by: java.lang.reflect.InvocationTargetException
> >>          at
> >> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> >> Method)
> >>          at
> >> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstruct
> >> or
> >> AccessorImpl.java:39)
> >>          at
> >> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingC
> >> on
> >> structorAccessorImpl.java:27)
> >>          at
> >> java.lang.reflect.Constructor.newInstance(Constructor.java:513)
> >>          at
> >> org.apache.hadoop.hbase.master.HMaster$3.run(HMaster.java:1238)
> >>          at
> >> org.apache.hadoop.hbase.master.HMaster$3.run(HMaster.java:1236)
> >>          ... 9 more
> >> Caused by: java.io.EOFException
> >>          at
> >> java.io.DataInputStream.readUnsignedShort(DataInputStream.java:323)
> >>          at java.io.DataInputStream.readUTF(DataInputStream.java:572)
> >>          at
> >> org.apache.hadoop.hbase.util.FSUtils.getVersion(FSUtils.java:151)
> >>          at
> >> org.apache.hadoop.hbase.util.FSUtils.checkVersion(FSUtils.java:170)
> >>          at
> >>
> org.apache.hadoop.hbase.master.HMaster.checkRootDir(HMaster.java:304)
> >>          at
> >> org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:221)
> >>          ... 15 more
> >


Mime
View raw message