hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Purtell (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-1918) Don't do DNS resolving in .META. scanner for each row
Date Tue, 20 Oct 2009 03:45:59 GMT

    [ https://issues.apache.org/jira/browse/HBASE-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12767675#action_12767675

Andrew Purtell commented on HBASE-1918:

The -v3 patch fixes the issues I'm seeing on 0.20 branch also. 

> Don't do DNS resolving in .META. scanner for each row
> -----------------------------------------------------
>                 Key: HBASE-1918
>                 URL: https://issues.apache.org/jira/browse/HBASE-1918
>             Project: Hadoop HBase
>          Issue Type: Improvement
>    Affects Versions: 0.19.3, 0.20.1
>            Reporter: Jean-Daniel Cryans
>            Assignee: Jean-Daniel Cryans
>             Fix For: 0.20.2, 0.21.0
>         Attachments: HBASE-1918-2.patch, HBASE-1918-3.patch, HBASE-1918.patch
> While debugging a slow HBase on my new Mac, I found that my issue was that DNS resolving
was taking 10 seconds. Since Java 1.5, many people complained about that and there's even
a very commented issue (http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=5092063) that's
closed but still "active". In fact the real issue here is that when you a bad DNS resolving
(my router was acting on me, it's dead now) it waits for 10 seconds or something like that
(see previous link). So I was running PE and my meta scanner was being slower and slower.
In fact, everything was slow, even starting the shell. So using jps I found that we are resolving
DNS for each scanned row in META in checkAssigned here:
> {code}
> if (sa != null && sa.length() > 0) {
>   serverName = HServerInfo.getServerName(sa, sc);
> } 
> HServerInfo storedInfo = null;
> {code}
> HSI.getServerName here creates a new HServerAddress which resolves DNS in its depths
and it's not even necessary, we use it just to split the server IP and port.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message