hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Xiang Hua <bea...@gmail.com>
Subject Re: hmaster and regionserver died
Date Mon, 15 Oct 2012 07:11:59 GMT
We will check the zk log.

On Monday, October 15, 2012, Ramkrishna.S.Vasudevan wrote:

> Check your GC configurations.  Seems to that a Full GC has happened and the
> Zookeeper thought that to be session expiry.
>
> Regards
> Ram
>
> > -----Original Message-----
> > From: Xiang Hua [mailto:beatls@gmail.com]
> > Sent: Saturday, October 13, 2012 6:20 PM
> > To: user@hbase.apache.org
> > Subject: hmaster and regionserver died
> >
> > Hi,
> >    the HMaster died as well as regionservers, below is hmaster's log.
> > could
> > you please find what's problem?
> >
> >
> > 2012-10-12 00:14:19,444 INFO org.apache.zookeeper.ClientCnxn: Socket
> > connection established to bj-ecsxhm4f3I-r3-5-r810-2-hbase-stor-3/
> > 10.20.16.34:2181, initiating session
> > 2012-10-12 00:14:19,520 INFO org.apache.zookeeper.ClientCnxn: Session
> > establishment complete on server bj-ecsxhm4f3I-r3-5-r810-2-hbase-stor-
> > 3/
> > 10.20.16.34:2181, sessionid = 0x139c539bc090002, negotiated timeout =
> > 40000
> > 2012-10-12 00:14:23,738 INFO org.apache.zookeeper.ClientCnxn: Client
> > session timed out, have not heard from server in 15046ms for sessionid
> > 0x239c539ba630001, closing socket connection and attempting reconnect
> > 2012-10-12 00:14:24,246 INFO org.apache.zookeeper.ClientCnxn: Opening
> > socket connection to server bj-ecsxhm4f3I-r3-5-r810-3-hbase-stor-2/
> > 10.20.16.33:2181
> > 2012-10-12 00:14:25,173 INFO org.apache.zookeeper.ClientCnxn: Client
> > session timed out, have not heard from server in 15245ms for sessionid
> > 0x139c539bc090003, closing socket connection and attempting reconnect
> > 2012-10-12 00:14:25,328 INFO org.apache.zookeeper.ClientCnxn: Opening
> > socket connection to server bj-ecsxhm4f3I-r3-5-r810-3-hbase-stor-2/
> > 10.20.16.33:2181
> > 2012-10-12 00:14:25,328 INFO org.apache.zookeeper.ClientCnxn: Socket
> > connection established to bj-ecsxhm4f3I-r3-5-r810-3-hbase-stor-2/
> > 10.20.16.33:2181, initiating session
> > 2012-10-12 00:14:25,507 INFO org.apache.zookeeper.ClientCnxn:
> > EventThread
> > shut down
> > 2012-10-12 00:14:25,507 INFO org.apache.zookeeper.ClientCnxn: Unable to
> > reconnect to ZooKeeper service, session 0x139c539bc090003 has expired,
> > closing socket connection
> > 2012-10-12 00:14:27,247 INFO org.apache.zookeeper.ClientCnxn: Socket
> > connection established to bj-ecsxhm4f3I-r3-5-r810-3-hbase-stor-2/
> > 10.20.16.33:2181, initiating session
> > 2012-10-12 00:14:27,248 WARN org.apache.zookeeper.ClientCnxn: Session
> > 0x239c539ba630001 for server bj-ecsxhm4f3I-r3-5-r810-3-hbase-stor-2/
> > 10.20.16.33:2181, unexpected error, closing socket connection and
> > attempting reconnect
> > java.io.IOException: Connection reset by peer
> >     at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
> >     at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
> >     at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:218)
> >     at sun.nio.ch.IOUtil.read(IOUtil.java:186)
> >     at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:359)
> >     at
> > org.apache.zookeeper.ClientCnxn$SendThread.doIO(ClientCnxn.java:859)
> >     at
> > org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1157)
> > 2012-10-12 00:14:28,026 INFO org.apache.zookeeper.ClientCnxn: Opening
> > socket connection to server bj-ecsxhm4f3I-r3-5-r810-2-hbase-stor-3/
> > 10.20.16.34:2181
> > 2012-10-12 00:14:41,359 INFO org.apache.zookeeper.ClientCnxn: Client
> > session timed out, have not heard from server in 14007ms for sessionid
> > 0x239c539ba630001, closing socket connection and attempting reconnect
> > 2012-10-12 00:14:41,592 INFO org.apache.zookeeper.ClientCnxn: Opening
> > socket connection to server bj-ecsxhm4f3I-r3-5-r810-4-hbase-stor-1/
> > 10.20.16.32:2181
> > 2012-10-12 00:14:46,186 INFO org.apache.zookeeper.ClientCnxn: Client
> > session timed out, have not heard from server in 26666ms for sessionid
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message