Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Content-Type: text/plain; charset=us-ascii
Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2104\))
Subject: Re: HBase on docker NotServingRegionException because of hostname alisas
From: Pierre Caserta <pierre.caserta@gmail.com>
In-Reply-To: <CAKYcnAtj9Tjeumr+5Mqo6fmCMHdjz9bLE_xx-xE24pgQL_sCWg@mail.gmail.com>
Date: Mon, 5 Sep 2016 17:04:09 +1000
Content-Transfer-Encoding: quoted-printable
Message-Id: <F785A6A2-4AA3-45F9-98A0-F250CB21F609@gmail.com>
References: <06089C15-4B61-46C4-A7F7-F11F866275B6@gmail.com> <CAKYcnAtj9Tjeumr+5Mqo6fmCMHdjz9bLE_xx-xE24pgQL_sCWg@mail.gmail.com>
To: user@hbase.apache.org
archived-at: Mon, 05 Sep 2016 07:01:06 -0000

That is a good script thanks but I would like to understand exactly what =
is the problem with my config without adding another level of =
abstraction and just running the clusterdock command.
In your script I can see that you are using --net=3Dhost. I think this =
is the main difference compared to what I am doing which is creating a =
bridge network for the hadoop cluster.
I have only 3 machines: hadoop-master, hadoop-slave1, hadoop-slave2.

Why do those strange hadoop-slave2.hadoopnet alias appear in the web ui?
It looks like the network name is used as part of the hostname.
Any idea what it is happening in my case?

Pierre

> On 5 Sep 2016, at 16:48, Dima Spivak <dimaspivak@apache.org> wrote:
>=20
> You should try the Apache HBase topology for clusterdock that was =
committed
> a few months back. See HBASE-12721 for details.
>=20
> On Sunday, September 4, 2016, Pierre Caserta =
<pierre.caserta@gmail.com>
> wrote:
>=20
>> Hi,
>> I am building a fully distributed hbase cluster with unmanaged =
zookeeper.
>> I pretty much used this example and install hbase on top of it:
>> https://github.com/kiwenlau/hadoop-cluster-docker
>>=20
>> Hadoop and hdfs works fine but I get this exception with hbase:
>>=20
>>    2016-09-05 06:27:12,268 INFO  =
[hadoop-master:16000.activeMasterManager]
>> zookeeper.MetaTableLocator: Failed verification of hbase:meta,,1 at
>> address=3Dhadoop-slave2,16020,1473052276351, =
exception=3Dorg.apache.hadoop.
>> hbase.NotServingRegionException: Region hbase:meta,,1 is not online =
on
>> hadoop-slave2.hadoopnet,16020,1473056813966
>>        at org.apache.hadoop.hbase.regionserver.HRegionServer.
>> getRegionByEncodedName(HRegionServer.java:2910)
>>=20
>> This is bloking because any command I enter on the hbase shell will =
return
>> the following error:
>>=20
>>    ERROR: org.apache.hadoop.hbase.PleaseHoldException: Master is
>> initializing
>>=20
>> The containers are runned using --net=3Dhadoopnet
>> which is a network create as such:
>>=20
>>    docker network create --driver=3Dbridge hadoopnet
>>=20
>> The hbase webui is showing this:
>>=20
>>    Region Servers
>>    ServerName  Start time      Version Requests Per Second     Num.
>> Regions
>>    hadoop-slave1,16020,1473056814064   Mon Sep 05 06:26:54 UTC 2016
>> 1.2.2   0       0
>>    hadoop-slave1.hadoopnet,16020,1473056814064 Mon Sep 05 06:26:54 =
UTC
>> 2016    Unknown 0       0
>>    hadoop-slave2,16020,1473056813966   Mon Sep 05 06:26:53 UTC 2016
>> 1.2.2   0       0
>>    hadoop-slave2.hadoopnet,16020,1473056813966 Mon Sep 05 06:26:53 =
UTC
>> 2016    Unknown 0       0
>>    Total:4             2 nodes with inconsistent version       0      =
 0
>>=20
>> I should have only 2 regionservers but 2 strange =
hadoop-slave1.hadoopnet
>> and hadoop-slave2.hadoopnet are added to the list.
>> When I look at zk using:
>>=20
>>    /usr/local/hbase/bin/hbase zkcli -server zk:2181 ls /hbase/rs
>>=20
>> I only see my 2 regionserver: hadoop-slave1,16020,1473056814064 and
>> hadoop-slave2,16020,1473056813966
>>=20
>> Looking at the zookeeper.MetaTableLocator: Failed verification error =
I see
>> that  hadoop-slave2,16020,1473052276351 and =
hadoop-slave2.hadoopnet,16020,1473056813966
>> get mixed up.
>>=20
>> here is my config on all server
>>=20
>>        <?xml version=3D"1.0" encoding=3D"UTF-8"?>
>>        <?xml-stylesheet type=3D"text/xsl" href=3D"configuration.xsl"?>
>>=20
>>        <configuration>
>>          <property>
>>                <name>hbase.rootdir</name>
>>            <value>hdfs://hadoop-master:9000/hbase</value>
>>              <description>The directory shared by region servers. =
Should
>> be fully-qualified to include the filesystem to use. E.g:
>> hdfs://NAMENODE_SERVER:PORT/HBASE_ROOTDIR</description>
>>          </property>
>>          <property>
>>              <name>hbase.master</name>
>>              <value>hdfs://hadoop-master:60000</value>
>>              <description>The host and port that the HBase master =
runs
>> at.</description>
>>          </property>
>>          <property>
>>              <name>hbase.cluster.distributed</name>
>>              <value>true</value>
>>              <description>The mode the cluster will be in. Possible
>> values are
>>              false: standalone and pseudo-distributed setups with =
managed
>> Zookeeper
>>              true: fully-distributed with unmanaged Zookeeper Quorum =
(see
>> hbase-env.sh)</description>
>>          </property>
>>          <property>
>>              <name>hbase.master.info.port</name>
>>              <value>60010</value>
>>              <description>The UI interface of HBase master
>> runs.</description>
>>          </property>
>>          <property>
>>              <name>hbase.zookeeper.quorum</name>
>>              <value>zk</value>
>>              <description>string m_e_m_b_e_r_s is replaced by list of
>> hosts separated by comma. Its generated by configure-slaves.sh on =
master
>> node</description>
>>          </property>
>>          <property>
>>              <name>hbase.zookeeper.property.maxClientCnxns</name>
>>              <value>300</value>
>>          </property>
>>          <property>
>>              <name>hbase.zookeeper.property.datadir</name>
>>              <value>/tmp/zookeeper</value>
>>              <description>location of storage of zookeeper
>> data</description>
>>          </property>
>>          <property>
>>              <name>hbase.zookeeper.property.clientPort</name>
>>              <value>2181</value>
>>          </property>
>>=20
>>        </configuration>
>>=20
>> I created a stack overflow question as well: =
http://stackoverflow.com/
>> questions/39325041/hbase-on-docker-notservingregionexception-
>> because-of-hostname-alisas <http://stackoverflow.com/
>> questions/39325041/hbase-on-docker-notservingregionexception-
>> because-of-hostname-alisas>
>>=20
>> Thanks,
>> Pierre
>=20
>=20
>=20
> --=20
> -Dima