hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ian Brooks <i.bro...@sensewhere.com>
Subject Re: HBase Master cannot be contacted by Region Servers
Date Wed, 09 Jul 2014 13:35:21 GMT
Hi,

Have you checked the master process is listening on port 60000 and that you can open a connection
to that port from the slave? could be firewalled off.

Also the regionserver process appears to be using an ip on a different subnet 10.0.2.15, are
there 2 interfaces on the slave?, if so you may want to add the slaves 192.168.66.61 address
to /etc/hosts

-Ian Brooks

On Wednesday 09 Jul 2014 15:29:44 Cosmin Cătălin Sanda wrote:
> The port should not be needed if the default settings have not been
> modified. I also don't see how that could be the problem since the
> default hbase created structure is visible under hdfs. I will try that
> nevertheless.
> ------------------------------------
> Cosmin Catalin SANDA
> Software Systems Engineer
> Phone: +45.27.30.60.35
> 
> 
> 
> On Wed, Jul 9, 2014 at 3:24 PM, Wellington Chevreuil
> <wellington.chevreuil@gmail.com> wrote:
> > Hi,
> >
> > Had you already tried to set the port on hbase.rootdir for hbase-site.xml, like
below:
> >
> >    <property>
> >        <name>hbase.rootdir</name>
> >        <value>hdfs://master:8020/hbase</value>
> >    </property>
> >
> > Cheers.
> >
> > On 9 Jul 2014, at 14:18, Cosmin Cătălin Sanda <cosmincatalin@gmail.com>
wrote:
> >
> >> I have a Hadoop cluster made of 3 slaves and 1 master on top of which
> >> there is an HBase cluster with 3 RS and 1 master respectively.
> >> Additionally there is a Zookeeper ensemble on 3 machines.
> >>
> >> The Hadoop cluster is functioning correctly as well as the Zookeeper
> >> ensemble. However, the HBase cluster fails to initialize correctly.
> >>
> >> I start HBase it by running ./bin/start-hbase.sh. This correctly
> >> starts the HBase Master and the Region Servers. The hbase folder in
> >> hdfs is set-up correctly.
> >>
> >> jps on master
> >>
> >> hduser@master:~/hbase$ jps
> >> 5694 HMaster
> >> 3934 JobHistoryServer
> >> 3786 NameNode
> >> 3873 ResourceManager
> >> 6025 Jps
> >>
> >> jps on slaves
> >>
> >> 5737 Jps
> >> 5499 HRegionServer
> >> 3736 DataNode
> >> 3820 NodeManager
> >>
> >> However, the HBase master does not register the Region Servers as it
> >> is also apparent from looking at the logs:
> >>
> >> master log
> >>
> >> [master:master:60000] master.ServerManager: Waiting for region servers
> >> count to settle; currently checked in 0, slept for 1511 ms, expecting
> >> minimum of 1, maximum of 2147483647, timeout of 4500 ms, interval of
> >> 1500 ms.
> >>
> >> slave log
> >>
> >> [regionserver60020] regionserver.HRegionServer: reportForDuty to
> >> master=master,60000,1404856451890 with port=60020,
> >> startcode=1404856453874
> >> [regionserver60020] regionserver.HRegionServer: error telling master we are
up
> >> com.google.protobuf.ServiceException:
> >> org.apache.hadoop.net.ConnectTimeoutException: 20000 millis timeout
> >> while waiting for channel to be ready for connect. ch :
> >> java.nio.channels.SocketChannel[connection-pending
> >> local=/10.0.2.15:53939 remote=master/192.168.66.60:60000]
> >>
> >> Here are the configuration details:
> >>
> >> /etc/hosts on master
> >>
> >> 192.168.66.63   slave-3 # Data Node and Region Server
> >> 192.168.66.60   master # Name Node and HBase Master
> >> 192.168.66.73   zookeeper-3 # Zookeeper node
> >> 192.168.66.71   zookeeper-1 # Zookeeper node
> >> 192.168.66.72   zookeeper-2 # Zookeeper node
> >> 192.168.66.62   slave-2 # Data Node and Region Server
> >> 192.168.66.61   slave-1 # Data Node and Region Server
> >>
> >> /etc/hosts on slave-1
> >>
> >> 192.168.66.60   master
> >> 192.168.66.73   zookeeper-3
> >> 192.168.66.71   zookeeper-1
> >> 192.168.66.72   zookeeper-2
> >>
> >> hbase-site.xml on ALL cluster nodes
> >>
> >> <?xml version="1.0"?>
> >> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
> >> <configuration>
> >>    <property>
> >>        <name>hbase.tmp.dir</name>
> >>        <value>/home/hduser/hbase/tmp</value>
> >>    </property>
> >>    <property>
> >>        <name>hbase.rootdir</name>
> >>        <value>hdfs://master/hbase</value>
> >>    </property>
> >>    <property>
> >>        <name>hbase.cluster.distributed</name>
> >>        <value>true</value>
> >>    </property>
> >>    <property>
> >>        <name>hbase.local.dir</name>
> >>        <value>/home/hduser/hbase/local</value>
> >>    </property>
> >>    <property>
> >>        <name>hbase.master.info.port</name>
> >>        <value>6010</value>
> >>    </property>
> >>    <property>
> >>        <name>hbase.zookeeper.quorum</name>
> >>        <value>zookeeper-1,zookeeper-2,zookeeper-3,</value>
> >>    </property>
> >> </configuration>
> >>
> >> regionservers file on master and slaves
> >>
> >> slave-3
> >> slave-1
> >> slave-2
> >>
> >> hbase-env.sh on master and slaves
> >>
> >> export JAVA_HOME=$(readlink -f /usr/bin/javac | sed "s:/bin/javac::"
> >> export HBASE_OPTS="-XX:+UseConcMarkSweepGC"
> >> export HBASE_MANAGES_ZK=false
> >>
> >> What am I doing wrong so that the nodes cannot talk to each other?
> >>
> >> I am using Hadoop 2.4.0 and HBase 0.98.3 along with Zookeeper 3.4.6 on
> >> Ubuntu Trusty Tahr x64.
> >

Mime
View raw message