hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jean-Daniel Cryans (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-1918) Don't do DNS resolving in .META. scanner for each row
Date Mon, 19 Oct 2009 20:04:59 GMT

    [ https://issues.apache.org/jira/browse/HBASE-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12767503#action_12767503

Jean-Daniel Cryans commented on HBASE-1918:

Thanks guys for looking at the patch.

Stack, here's the code in HSA that was called:


    int colonIndex = hostAndPort.lastIndexOf(':');
    if(colonIndex < 0) {
      throw new IllegalArgumentException("Not a host:port pair: " + hostAndPort);
    String host = hostAndPort.substring(0, colonIndex);
    int port =
      Integer.valueOf(hostAndPort.substring(colonIndex + 1)).intValue();

At first I tried to put it in a helper method but the code is so simple I could just copy
it over so we'll have the same behavior.

> Don't do DNS resolving in .META. scanner for each row
> -----------------------------------------------------
>                 Key: HBASE-1918
>                 URL: https://issues.apache.org/jira/browse/HBASE-1918
>             Project: Hadoop HBase
>          Issue Type: Improvement
>    Affects Versions: 0.19.3, 0.20.1
>            Reporter: Jean-Daniel Cryans
>            Assignee: Jean-Daniel Cryans
>             Fix For: 0.20.2, 0.21.0
>         Attachments: HBASE-1918.patch
> While debugging a slow HBase on my new Mac, I found that my issue was that DNS resolving
was taking 10 seconds. Since Java 1.5, many people complained about that and there's even
a very commented issue (http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=5092063) that's
closed but still "active". In fact the real issue here is that when you a bad DNS resolving
(my router was acting on me, it's dead now) it waits for 10 seconds or something like that
(see previous link). So I was running PE and my meta scanner was being slower and slower.
In fact, everything was slow, even starting the shell. So using jps I found that we are resolving
DNS for each scanned row in META in checkAssigned here:
> {code}
> if (sa != null && sa.length() > 0) {
>   serverName = HServerInfo.getServerName(sa, sc);
> } 
> HServerInfo storedInfo = null;
> {code}
> HSI.getServerName here creates a new HServerAddress which resolves DNS in its depths
and it's not even necessary, we use it just to split the server IP and port.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message