hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jean-Daniel Cryans (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-1918) Don't do DNS resolving in .META. scanner for each row
Date Mon, 19 Oct 2009 20:04:59 GMT

    [ https://issues.apache.org/jira/browse/HBASE-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12767503#action_12767503
] 

Jean-Daniel Cryans commented on HBASE-1918:
-------------------------------------------

Thanks guys for looking at the patch.

Stack, here's the code in HSA that was called:

{code}

    int colonIndex = hostAndPort.lastIndexOf(':');
    if(colonIndex < 0) {
      throw new IllegalArgumentException("Not a host:port pair: " + hostAndPort);
    }
    String host = hostAndPort.substring(0, colonIndex);
    int port =
      Integer.valueOf(hostAndPort.substring(colonIndex + 1)).intValue();
{code}

At first I tried to put it in a helper method but the code is so simple I could just copy
it over so we'll have the same behavior.

> Don't do DNS resolving in .META. scanner for each row
> -----------------------------------------------------
>
>                 Key: HBASE-1918
>                 URL: https://issues.apache.org/jira/browse/HBASE-1918
>             Project: Hadoop HBase
>          Issue Type: Improvement
>    Affects Versions: 0.19.3, 0.20.1
>            Reporter: Jean-Daniel Cryans
>            Assignee: Jean-Daniel Cryans
>             Fix For: 0.20.2, 0.21.0
>
>         Attachments: HBASE-1918.patch
>
>
> While debugging a slow HBase on my new Mac, I found that my issue was that DNS resolving
was taking 10 seconds. Since Java 1.5, many people complained about that and there's even
a very commented issue (http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=5092063) that's
closed but still "active". In fact the real issue here is that when you a bad DNS resolving
(my router was acting on me, it's dead now) it waits for 10 seconds or something like that
(see previous link). So I was running PE and my meta scanner was being slower and slower.
In fact, everything was slow, even starting the shell. So using jps I found that we are resolving
DNS for each scanned row in META in checkAssigned here:
> {code}
> if (sa != null && sa.length() > 0) {
>   serverName = HServerInfo.getServerName(sa, sc);
> } 
> HServerInfo storedInfo = null;
> {code}
> HSI.getServerName here creates a new HServerAddress which resolves DNS in its depths
and it's not even necessary, we use it just to split the server IP and port.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message