hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jean-Daniel Cryans (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HBASE-1918) Don't do DNS resolving in .META. scanner for each row
Date Mon, 19 Oct 2009 20:24:59 GMT

     [ https://issues.apache.org/jira/browse/HBASE-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Jean-Daniel Cryans updated HBASE-1918:

    Attachment: HBASE-1918-2.patch

Something like this patch.

> Don't do DNS resolving in .META. scanner for each row
> -----------------------------------------------------
>                 Key: HBASE-1918
>                 URL: https://issues.apache.org/jira/browse/HBASE-1918
>             Project: Hadoop HBase
>          Issue Type: Improvement
>    Affects Versions: 0.19.3, 0.20.1
>            Reporter: Jean-Daniel Cryans
>            Assignee: Jean-Daniel Cryans
>             Fix For: 0.20.2, 0.21.0
>         Attachments: HBASE-1918-2.patch, HBASE-1918.patch
> While debugging a slow HBase on my new Mac, I found that my issue was that DNS resolving
was taking 10 seconds. Since Java 1.5, many people complained about that and there's even
a very commented issue (http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=5092063) that's
closed but still "active". In fact the real issue here is that when you a bad DNS resolving
(my router was acting on me, it's dead now) it waits for 10 seconds or something like that
(see previous link). So I was running PE and my meta scanner was being slower and slower.
In fact, everything was slow, even starting the shell. So using jps I found that we are resolving
DNS for each scanned row in META in checkAssigned here:
> {code}
> if (sa != null && sa.length() > 0) {
>   serverName = HServerInfo.getServerName(sa, sc);
> } 
> HServerInfo storedInfo = null;
> {code}
> HSI.getServerName here creates a new HServerAddress which resolves DNS in its depths
and it's not even necessary, we use it just to split the server IP and port.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message