hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nick Dimiduk <ndimi...@gmail.com>
Subject Re: 答复: how config multi regionserver, or what is wrong?
Date Tue, 11 Dec 2012 02:36:58 GMT
For starters, you'll need to sync up the clocks on all your machines.
Install ntp or similar and those ClockOutOfSync exceptions will clear up.
Specifically, blade4 looks off.

-n
On Dec 10, 2012 6:18 PM, "tgh" <guanhua.tian@ia.ac.cn> wrote:

> Meanwhile , log is master , that is, blade1, is like this,  there are some
> ERRor like this, for
>
> 2012-09-01 06:31:05,558 INFO org.apache.hadoop.hbase.master.ServerManager:
> Registering server=blade2,60020,1346452716636, regionCount=0, userLoad=false
> 2012-09-01 06:31:05,569 WARN org.apache.hadoop.hbase.master.ServerManager:
> Server blade4,60020,1346451768443 has been rejected; Reported time is too
> far out of sync with master.  Time difference of 496371ms > max allowed of
> 30000ms
> 2012-09-01 06:31:05,581 WARN org.apache.hadoop.hbase.master.ServerManager:
> Server blade5,60020,1346452001672 has been rejected; Reported time is too
> far out of sync with master.  Time difference of 263137ms > max allowed of
> 30000ms
> 2012-09-01 06:31:05,583 ERROR org.apache.hadoop.hbase.master.HMaster:
> Region server serverName=blade4,60020,1346451768443, load=(requests=0,
> regions=0, usedHeap=142, maxHeap=966) reported a fatal error:
> ABORTING region server serverName=blade4,60020,1346451768443,
> load=(requests=0, regions=0, usedHeap=142, maxHeap=966): Unhandled
> exception: org.apache.hadoop.hbase.ClockOutOfSyncException: Server
> blade4,60020,1346451768443 has been rejected; Reported time is too far out
> of sync with master.  Time difference of 496371ms > max allowed of 30000ms
> Cause:
> org.apache.hadoop.hbase.ClockOutOfSyncException:
> org.apache.hadoop.hbase.ClockOutOfSyncException: Server
> blade4,60020,1346451768443 has been rejected; Reported time is too far out
> of sync with master.  Time difference of 496371ms > max allowed of 30000ms
>         at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
>         at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
>         at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>         at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
>         at
> org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:95)
>         at
> org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:79)
>         at
> org.apache.hadoop.hbase.regionserver.HRegionServer.reportForDuty(HRegionServer.java:1574)
>         at
> org.apache.hadoop.hbase.regionserver.HRegionServer.tryReportForDuty(HRegionServer.java:1531)
>         at
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:572)
>         at java.lang.Thread.run(Thread.java:722)
> Caused by: org.apache.hadoop.ipc.RemoteException:
> org.apache.hadoop.hbase.ClockOutOfSyncException: Server
> blade4,60020,1346451768443 has been rejected; Reported time is too far out
> of sync with master.  Time difference of 496371ms > max allowed of 30000ms
>         at
> org.apache.hadoop.hbase.master.ServerManager.checkClockSkew(ServerManager.java:193)
>         at
> org.apache.hadoop.hbase.master.ServerManager.regionServerStartup(ServerManager.java:141)
>         at
> org.apache.hadoop.hbase.master.HMaster.regionServerStartup(HMaster.java:675)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>         at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:601)
>         at
> org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:570)
>         at
> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1039)
>
>         at
> org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:771)
>         at
> org.apache.hadoop.hbase.ipc.HBaseRPC$Invoker.invoke(HBaseRPC.java:257)
>         at $Proxy5.regionServerStartup(Unknown Source)
>         at
> org.apache.hadoop.hbase.regionserver.HRegionServer.reportForDuty(HRegionServer.java:1570)
>         ... 3 more
>
>
> -----邮件原件-----
> 发件人: user-return-32385-guanhua.tian=ia.ac.cn@hbase.apache.org [mailto:
> user-return-32385-guanhua.tian=ia.ac.cn@hbase.apache.org] 代表 tgh
> 发送时间: 2012年12月11日 9:59
> 收件人: user@hbase.apache.org
> 主题: 答复: how config multi regionserver, or what is wrong?
>
> And our hosts is follows
>
> [root@blade1 ~]# cat /etc/hosts
> 127.0.0.1   localhost localhost.localdomain localhost4
> localhost4.localdomain4
> ::1         localhost localhost.localdomain localhost6
> localhost6.localdomain6
>
> 192.168.76.233 blade1
> 192.168.76.234 blade2
> 192.168.76.235 blade3
> 192.168.76.236 blade4
> 192.168.76.237 blade5
> 192.168.76.238 blade6
> 192.168.76.239 blade7
> 192.168.76.240 blade8
>
> 192.168.76.245 fnode1
> 192.168.76.246 fnode2
> [root@blade1 ~]#
>
> -----邮件原件-----
> 发件人: user-return-32384-guanhua.tian=ia.ac.cn@hbase.apache.org [mailto:
> user-return-32384-guanhua.tian=ia.ac.cn@hbase.apache.org] 代表 tgh
> 发送时间: 2012年12月11日 9:00
> 收件人: user@hbase.apache.org
> 主题: 答复: how config multi regionserver, or what is wrong?
>
> Thank you for your reply,
> And the configuration file is here,
> Could you help me,
>
>
> Thank you
> ---------------------------
> Tian Guanhua
>
>
>
> [root@blade1 conf]# cat regionservers
> blade1
> blade2
> blade3
> blade4
> blade5
> blade6
> blade7
> blade8
> [root@blade1 conf]#
> [root@blade1 conf]# vim hbase-site.xml
> <?xml version="1.0"?>
> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
> <!--
> -->
> <configuration>
> <property>
>     <name>hbase.rootdir</name>
>     <value>hdfs://blade1:9000/hbase</value>
>     <description>The directory shared by RegionServers.</description>
> </property>
> <property>
>     <name>hbase.cluster.distributed</name>
>     <value>true</value>
> </property>
> <property>
>     <name>hbase.zookeeper.quorum</name>
>     <value>blade1,blade2,blade3</value>
> </property>
>  <property>
>     <name>hbase.zookeeper.property.dataDir</name>
>    <value>/home/liuxin/zookeeper/data</value>
>  </property>
>  <property>
>   <name>dfs.support.append</name>
>    <value>true</value>
>  </property>
>  <property>
>   <name>dfs.datanode.max.xcievers</name>
>    <value>4096</value>
>    </property>
>  <property>
>  <name>hbase.master</name>
>  <value>blade1:60000</value>
>  </property>
> </configuration>
>
>
>
>
>
>
> -----邮件原件-----
> 发件人: user-return-32370-guanhua.tian=ia.ac.cn@hbase.apache.org [mailto:
> user-return-32370-guanhua.tian=ia.ac.cn@hbase.apache.org] 代表 Jean-Marc
> Spaggiari
> 发送时间: 2012年12月10日 20:54
> 收件人: user@hbase.apache.org
> 主题: Re: how config multi regionserver, or what is wrong?
>
> Hi Tian,
>
> Can you share you configuration files?
>
> Do you have something like that on your hbase-site.xml file?
>
>   <property>
>     <name>hbase.cluster.distributed</name>
>     <value>true</value>
>     <description>The mode the cluster will be in. Possible values are
>       false: standalone and pseudo-distributed setups with managed
> Zookeeper
>       true: fully-distributed with unmanaged Zookeeper Quorum (see
> hbase-env.sh)
>     </description>
>   </property>
>
>
> JM
>
> 2012/12/10, tgh <guanhua.tian@ia.ac.cn>:
> > Hi
> >       I try to use hbase, and now ,I have a problem with hbase
> > configuration, I use 8node for try, and it seems to work, hadoop,
> > zookeeper, hbase all boot up, and it could insert into with putAPI ,
> > But when I try to use masterIP:60010 to manage it, I find there are
> > only one regionserver there, and it is localhost:60030, why?
> >       I have set regionserver file , there are 8 nodes there, and all
> could
> > ssh to each other without passwd,
> >
> >       And I also use hdfsNamenodeIP:50070 to see HDFS, and it seems OK,
> the
> > data have been balanced across 8node, but I wander hbase only have one
> > regionserver to work, although when I star/stop regionserver , there
> > are 8region server to start and stop,
> >
> >       And I try to put data into hbase, it seems ok at first, but after
> > 200million in hbase, it seems really hard to insert more into it, it
> > is very slow, and I use masterIP:60010 to manage it, I find there are
> > only one regionserver there, and it is localhost:60030, why?
> >
> >
> >       Could you help me,
> >
> >
> >
> > Thank you
> > -------------------
> > Tian Guanhua
> >
> >
> >
> >
> >
> >
> >
>
>
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message