hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Duo Xu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-18226) Disable reverse DNS lookup at HMaster and use default hostname provided by RegionServer
Date Sun, 18 Jun 2017 06:39:01 GMT

    [ https://issues.apache.org/jira/browse/HBASE-18226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16053081#comment-16053081

Duo Xu commented on HBASE-18226:


Thanks for reviewing. Just realize HBASE-12953 was fixed by you :)

Can you come up with better name ?
When reverse DNS is enabled, resolution is also automatic.
How about hbase.regionserver.hostname.from.etc.hosts ?

I agree that the configuration name is not so accurate. 

I read the source code again (including hadoop-common dns.java). By default RS gets hostname
by calling InetAddress.getLocalHost().getCanonicalHostName(). If users specify "hbase.regionserver.dns.interface"
or "hbase.regionserver.dns.nameserver" or some underlying system configuration changes (eg.
modifying /etc/nsswitch.conf), it may first read from DNS or other sources instead of first
checking /etc/hosts file.

However, whether RS reads hostname from DNS or /etc/hosts is not a concern here. This JIRA
is to address that HMaster uses the hostname passed from RS instead of doing reverse DNS lookup
to tells RS which hostname to use. This has already been implemented by HBASE-12954 by adding
"useThisHostnameInstead" field in RegionServerStatusProtos.

Currently "useThisHostnameInstead" is optional and RS by default only passes port, server
start code and server current time info to HMaster during reportForDuty(). I would like to
introduce a setting that if the setting is set to true, "useThisHostnameInstead" will be set
to the hostname RS gets from rpcServices.isa.getHostName();. Then HMaster will skip reverse
DNS lookup because it sees "useThisHostnameInstead" field is set in the request.

How about hbase.regionserver.report.hostname.to.master?

I will update the JIRA description to make the goal more clearly. It seems I made people feel
confused :)

Is it possible to add unit test ?

The patch reuses "useThisHostnameInstead" field in RegionServerStatusProtos, so there is not
much to test. I will add a test like org.apache.hadoop.hbase.regionserver.TestRegionServerHostname.testRegionServerHostname,
with my configuration set to true, and verify e2e working by using the MiniCluster.

> Disable reverse DNS lookup at HMaster and use default hostname provided by RegionServer
> ---------------------------------------------------------------------------------------
>                 Key: HBASE-18226
>                 URL: https://issues.apache.org/jira/browse/HBASE-18226
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Duo Xu
>         Attachments: HBASE-18226.001.patch, HBASE-18226.002.patch
> This JIRA is to address the similar problem as HBASE-12954, but there are some little
> 1. HBASE-12954 provides the configuration "hbase.regionserver.hostname" so users can
configure it on every regionserver with preferred hostnames. However, this configuration cannot
be set through Ambari or other configuration management tools because each regionserver has
a different value of this setting, which means each node needs a different hbase-site.xml.
> 2. In Azure HDInsight clusters, we want to give each RegionServer/workernode a FQDN by
modifying /etc/hosts on that node. We do not want HMaster to do reverse DNS lookup because
HMaster VM's /etc/hosts does not have regionserver VM's FQDN mappings. In current implementation
when regionserver starts, 
> {code}
> String hostName = shouldUseThisHostnameInstead() ? useThisHostnameInstead :
>       rpcServices.isa.getHostName();
> {code}
> it uses FQDN names here but on HMaster side, it will do reverse DNS lookup which cannot
be resolved.
> {code}
>  // if regionserver passed hostname to use,
>  // then use it instead of doing a reverse DNS lookup
>  ServerName rs = master.getServerManager().regionServerStartup(request, ia);
> {code}
> My proposed fix is to add a new configuration "*hbase.regionserver.hostname.auto*". If
it is set to true, then Regionserver will use the value returned by *rpcServices.isa.getHostName()*
as the hostname overwriting whatever users specifies in "*hbase.regionserver.hostname*" and
send to HMaster as part of RegionServerStartupRequest. HMaster will not do reverse DNS lookup,
which has been implemented in HBASE-12954. If users want to provide their own hostnames in
"*hbase.regionserver.hostname*", "*hbase.regionserver.hostname.auto*" must be false.
> I will submit a patch later today.

This message was sent by Atlassian JIRA

View raw message