hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dima Spivak <dimaspi...@apache.org>
Subject Re: hbase has problems with two hostname
Date Tue, 17 Jan 2017 15:47:48 GMT
Is there any other DNS server running that might be confusing reverse
lookup? What happens if you run `host YOUR_RS_IP_ADDRESS`?

And what kind of machines are you using in your deployment?

Cheers,

On Mon, Jan 16, 2017 at 11:34 PM C R <cuirong198610@hotmail.com> wrote:

> Thanks,
>
>
>
>
>
> I deployed my HBase very simply, which has one Master and three
> regionservers.
>
>
>
>
>
> [hbase@bjsh19-16-30 conf]$ more regionservers
>
> bjsh19-16-33.qbos.com
>
> bjsh19-16-34.qbos.com
>
> bjsh19-16-35.qbos.com
>
> [hbase@bjsh19-16-30 conf]$ more hbase-site.xml
>
>
>
> ...
>
>
>
> <configuration>
>
> <property>
>
>         <name>zookeeper.znode.parent</name>
>
>                 <value>/hbase117</value>
>
>                 </property>
>
> <property>
>
>   <name>hbase.rootdir</name>
>
>         <value>hdfs://bidc/hbase117</value>
>
>                   </property>
>
> <property>
>
>   <name>hbase.zookeeper.quorum</name>
>
>         <value>bjsh19-16-30.qbos.com,bjsh19-16-31.qbos.com,
> bjsh19-16-32.qbos.com</value>
>
>                   </property>
>
> <property>
>
>   <name>hbase.cluster.distributed</name>
>
>         <value>true</value>
>
>                   </property>
>
> <property>
>
>   <name>hbase.zookeeper.property.clientPort</name>
>
>         <value>2181</value>
>
>                   </property>
>
> </configuration>
>
>
>
>
>
> The special place is the file /etc/hosts with one ip mapping to two
> hostnames on all nodes,so it will have the message:
>
>
>
> ...
>
>
>
> the server that tried to transition was wjsa-tsl05,16020,1484623636195 not
> the expected bjsh19-16-34.qbos.com,16020,1484623636195
>
>
>
> ...
>
>
>
>
>
> ________________________________
>
> 发件人: Dima Spivak <dimaspivak@apache.org>
>
> 发送时间: 2017年1月17日 4:50
>
> 收件人: user@hbase.apache.org
>
> 主题: Re: hbase has problems with two hostname
>
>
>
> Hi C R,
>
>
>
> Like many Hadoop-like services, HBase is pretty temperamental about
>
> requiring forward and reverse DNS to work properly. FWIW, the configuration
>
> file where you can populate RegionServers doesn't tend to matter as long as
>
> the hbase-site.xml file is populated correctly (it's just used to start
>
> daemons from one place).
>
>
>
> If you pass along more details about how exactly you're deploying HBase, we
>
> might be able to give more advice.
>
>
>
> On Mon, Jan 16, 2017 at 8:00 PM C R <cuirong198610@hotmail.com> wrote:
>
>
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> > more /etc/hosts
>
> >
>
> >
>
> > ...
>
> >
>
> >
>
> >
>
> >
>
> > 10.19.16.31  bjsh19-16-31.qbos.com    wjsa-tsl02
>
> >
>
> >
>
> > ...
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> > There will have six regionservers listed in web console, but
>
> >
>
> > only three in the configuration file,  metadata tables also are not
> online
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> > Hmaster will be dead after a while.
>
> >
>
> >
>
> > what should I do?
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> > snapshot:
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> > 2017-01-17 11:45:24,394 INFO
>
> >  [MASTER_SERVER_OPERATIONS-bjsh19-16-30:16000-0]
> master.AssignmentManager:
>
> > Assigning
> hbase:namespace,,1484623643279.30fab746cb3b6ceadcbda421459204b9.
>
> > to bjsh19-16-34.qbos.com,16020,1484623636195
>
> >
>
> >
>
> > 2017-01-17 11:45:24,395 INFO  [bjsh19-16-30:16000.activeMasterManager]
>
> > master.AssignmentManager: Joined the cluster in 23ms, failover=true
>
> >
>
> >
>
> > 2017-01-17 11:50:24,314 FATAL [bjsh19-16-30:16000.activeMasterManager]
>
> > master.HMaster: Failed to become active master
>
> >
>
> >
>
> > java.io.IOException: Timedout 300000ms waiting for namespace table to be
>
> > assigned
>
> >
>
> >
>
> >         at
>
> >
> org.apache.hadoop.hbase.master.TableNamespaceManager.start(TableNamespaceManager.java:104)
>
> >
>
> >
>
> >         at
>
> > org.apache.hadoop.hbase.master.HMaster.initNamespace(HMaster.java:986)
>
> >
>
> >
>
> >         at
>
> >
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:780)
>
> >
>
> >
>
> >         at
>
> > org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:183)
>
> >
>
> >
>
> >         at
> org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1652)
>
> >
>
> >
>
> >         at java.lang.Thread.run(Thread.java:745)
>
> >
>
> >
>
> > 2017-01-17 11:50:24,315 FATAL [bjsh19-16-30:16000.activeMasterManager]
>
> > master.HMaster: Master server abort: loaded coprocessors are: []
>
> >
>
> >
>
> > 2017-01-17 11:50:24,316 FATAL [bjsh19-16-30:16000.activeMasterManager]
>
> > master.HMaster: Unhandled exception. Starting shutdown.
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> > ...
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> > 2017-01-17 11:27:17,926 INFO  [regionserver/
>
> > bjsh19-16-34.qbos.com/10.19.16.34:16020] regionserver.HRegionServer:
>
> > Serving as wjsa-tsl05,16020,1
>
> >
>
> >
>
> > 484623636195, RpcServer on bjsh19-16-34.qbos.com/10.19.16.34:16020,
>
> > sessionid=0x154563e43e30179
>
> >
>
> >
>
> > 2017-01-17 11:27:17,934 INFO  [regionserver/
>
> > bjsh19-16-34.qbos.com/10.19.16.34:16020]
> quotas.RegionServerQuotaManager:
>
> > Quota support disabled
>
> >
>
> >
>
> > 2017-01-17 11:27:23,966 INFO
>
> >  [PriorityRpcServer.handler=14,queue=0,port=16020]
>
> > regionserver.RSRpcServices: Open hbase:namespace,,148462364327
>
> >
>
> >
>
> > 9.30fab746cb3b6ceadcbda421459204b9.
>
> >
>
> >
>
> > 2017-01-17 11:27:24,008 WARN  [RS_OPEN_REGION-bjsh19-16-34:16020-0]
>
> > zookeeper.ZKAssign: regionserver:16020-0x154563e43e30179,
> quorum=bjsh19-16
>
> >
>
> >
>
> > -30:2181,bjsh19-16-31:2181,bjsh19-16-32:2181, baseZNode=/hbase115new
>
> > Attempt to transition the unassigned node for 30fab746cb3b6ceadcbda421459
>
> >
>
> >
>
> > 204b9 from M_ZK_REGION_OFFLINE to RS_ZK_REGION_OPENING failed, the server
>
> > that tried to transition was wjsa-tsl05,16020,1484623636195 not the
>
> >
>
> >
>
> > expected bjsh19-16-34.qbos.com,16020,1484623636195
>
> >
>
> >
>
> > 2017-01-17 11:27:24,008 WARN  [RS_OPEN_REGION-bjsh19-16-34:16020-0]
>
> > coordination.ZkOpenRegionCoordination: Failed transition from OFFLINE to
> O
>
> >
>
> >
>
> > PENING for region=30fab746cb3b6ceadcbda421459204b9
>
> >
>
> >
>
> > 2017-01-17 11:27:24,008 WARN  [RS_OPEN_REGION-bjsh19-16-34:16020-0]
>
> > handler.OpenRegionHandler: Region was hijacked? Opening cancelled for
> enco
>
> >
>
> >
>
> > dedName=30fab746cb3b6ceadcbda421459204b9
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message