hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "liubangchen (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-18124) Make Hbase Communication Support Virtual Network
Date Sat, 27 May 2017 15:25:04 GMT

     [ https://issues.apache.org/jira/browse/HBASE-18124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

liubangchen updated HBASE-18124:
--------------------------------
    Description: 
Hbase only have one way to locate hmaster or hregionserver not like hdfs has two way to locate
datanode used by name or hostname.

I'm a engineer of cloud computing , and I'm in charge of to make Hbase as a cloud service.
Our Hbase cloud service architecture shown as follows:  1.jpg

1.VM
    User’s Hbase client work in vm and use virtual ip address to access hbase cluster.
2.NAT
    Network Address Translation, vip(Virtual Network Address) to pip (Physical Network Address)
3. HbaseCluster Service
     HbaseCluster Service work in physical network

Problem
     Vm use vip to communicate with hbase cluster, but hbase have only one way to communicate
with each server using host which is setting by parameters hbase.regionserver.hostname or
hbase.master.hostname. When HMaster startup will publish master address and meta region server
address  in zookeeper, then the address is pip(Physical Network Address) . Because hbase cluster
work in physical network. But the problem is that when vm get  the address from zookeeper
is pip, not vip. If I set host as vip, it will has problems for hbase cluster communication
inside. so it needs two ways for communication setting by parameters.

Solution
    1.   protocol extend 
      change strcut ServerName as
      {code}
          message ServerName {
              required string host_name = 1;
             optional uint32 port = 2;
             optional uint64 start_code = 3;
              optional string name=4;  //new field 
         }
      {code}
     It will be publish in zookeeper. We can choose host_name or name configured by parameters
hbase.client.use.hostname

2.   metatable extend 
       Add a column to hbase:meta named info:namelocation . So the original column info:server
configured with hbase.regionserver.hostname, and the new column info:namelocation  configured
with hbase.regionserver.servername
3.   hbase-server
       When regionserver startup, configured  hbase.regionserver.hostname as pip and configured
hbase.regionserver.servername as vip. Then hbase.regionserver.hostname will be writed in ServerName's
host_name, and hbase.regionserver.servername will be writed in ServerName's name.When hmaster
startup, configured hbase.hmaster.hostname as pip and configured  hbase.hmaster.servername
as vip. Then hbase.hmaster.hostname will be writed in ServerName's host_name, and hbase.hmaster.servername
will be writed in ServerName's name.
4.   hbase-client
      Add a parameters named hbase.client.use.hostname to use vip or pip.

This patch is base on Hbase-1.3.0

  was:
Hbase only have one way to locate hmaster or hregionserver not like hdfs has two way to locate
datanode use by name or hostname.

I’m a engineer of  cloud computing , and I’m in charge of to make Hbase as a cloud service,when
we make hbase as a cloud service we need  hbase support other way to support locate hmaster
or hregionserver
Our Hbase cloud service architectue shown as follows 1.jpg

1.VM
    User’s Hbase client work in vm and use virtual ip address to access hbase cluster.
2.NAT
   Network Address Translation, vip(Virtual Network Address) to pip (Physical Network Address)
3. HbaseCluster Service
     HbaseCluster Service work in physical network

Problem
1.  View on vm
          On vm side vm use vip to communication,but hbase have only one way to communication
use struct named
      ServerName. When Hmaster startup will store master address and meta region server address
in zookeeper, 
       then the address is pip(Physical Network Address)   because hbase cluster work in physical
network . when vm 
      get the address from zookeeper will not work because   vm use vip to communication,one
way to  solve this is to 
      make physical machine host as vip like 192.168.0.1,but is not better to make this.
2.  View on Physical machine
    Physical machine use pip to communication

Solution
    1.   protocol extend change proto message to below:
      {code}
          message ServerName {
              required string host_name = 1;
             optional uint32 port = 2;
             optional uint64 start_code = 3;
              optional string name=4;
         }
      {code}

     add a filed named name like hdfs’s datablock location
2.   metatable extend 
       add column to hbase:meta named info:namelocation
3.   hbase-server
      add params 
     {code}
      hbase.regionserver.servername
      <property>
            <name>hbase.regionserver.servername</name>
            <value>10.0.1.1</value>
     </property>
      {code}
      to regionserver namelocation
      add params
     {code}
       hbase.master.servername 
       <property>
           <name>hbase.master.servername</name>
           <value>10.0.1.2</value>
       </property>
     {code}
       to set master namelocation
4.   hbase-client
      add params 
    {code}
     hbase.client.use.hostname 
     <property>
         <name>hbase.client.use.hostname</name>
         <value>true</value>
     </property>
    {code}
     to choose which address to use

This patch is base on Hbase-1.3.0


> Make Hbase Communication Support Virtual Network
> ------------------------------------------------
>
>                 Key: HBASE-18124
>                 URL: https://issues.apache.org/jira/browse/HBASE-18124
>             Project: HBase
>          Issue Type: New Feature
>          Components: Client, hbase, master
>            Reporter: liubangchen
>            Assignee: liubangchen
>         Attachments: 1.jpg, HBASE-18124.patch, HBASE-18124.pdf
>
>
> Hbase only have one way to locate hmaster or hregionserver not like hdfs has two way
to locate datanode used by name or hostname.
> I'm a engineer of cloud computing , and I'm in charge of to make Hbase as a cloud service.
> Our Hbase cloud service architecture shown as follows:  1.jpg
> 1.VM
>     User’s Hbase client work in vm and use virtual ip address to access hbase cluster.
> 2.NAT
>     Network Address Translation, vip(Virtual Network Address) to pip (Physical Network
Address)
> 3. HbaseCluster Service
>      HbaseCluster Service work in physical network
> Problem
>      Vm use vip to communicate with hbase cluster, but hbase have only one way to communicate
with each server using host which is setting by parameters hbase.regionserver.hostname or
hbase.master.hostname. When HMaster startup will publish master address and meta region server
address  in zookeeper, then the address is pip(Physical Network Address) . Because hbase cluster
work in physical network. But the problem is that when vm get  the address from zookeeper
is pip, not vip. If I set host as vip, it will has problems for hbase cluster communication
inside. so it needs two ways for communication setting by parameters.
> Solution
>     1.   protocol extend 
>       change strcut ServerName as
>       {code}
>           message ServerName {
>               required string host_name = 1;
>              optional uint32 port = 2;
>              optional uint64 start_code = 3;
>               optional string name=4;  //new field 
>          }
>       {code}
>      It will be publish in zookeeper. We can choose host_name or name configured by parameters
hbase.client.use.hostname
> 2.   metatable extend 
>        Add a column to hbase:meta named info:namelocation . So the original column info:server
configured with hbase.regionserver.hostname, and the new column info:namelocation  configured
with hbase.regionserver.servername
> 3.   hbase-server
>        When regionserver startup, configured  hbase.regionserver.hostname as pip and
configured hbase.regionserver.servername as vip. Then hbase.regionserver.hostname will be
writed in ServerName's host_name, and hbase.regionserver.servername will be writed in ServerName's
name.When hmaster startup, configured hbase.hmaster.hostname as pip and configured  hbase.hmaster.servername
as vip. Then hbase.hmaster.hostname will be writed in ServerName's host_name, and hbase.hmaster.servername
will be writed in ServerName's name.
> 4.   hbase-client
>       Add a parameters named hbase.client.use.hostname to use vip or pip.
> This patch is base on Hbase-1.3.0



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message