hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Raghu Angadi (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-803) Reducing memory consumption on Namenode : Part 1
Date Mon, 22 Jan 2007 20:09:30 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12466546
] 

Raghu Angadi commented on HADOOP-803:
-------------------------------------

The bug is in the following patch. Very costly oversight: The does not affect equals() but
affects Block.comparedTo().

     public int compareTo(Object o) {
-        Block b = (Block) o;
-        if (getBlockId() < b.getBlockId()) {
-            return -1;
-        } else if (getBlockId() == b.getBlockId()) {
-            return 0;
-        } else {
-            return 1;
-        }
+        long diff = getBlockId() - ((Block)o).getBlockId();
+        return ( diff < 0 ) ? -1 : ( ( diff > 0 ) ? 1 : 0 );
     }

e.g: 'diff' wont be < 0 when blocks ids are LONG_MAX and -10.

changing this to following fixes it. 

+        Block b = (Block)o;
+        return ( blkid < b.blkid ) ? -1 :
+               ( ( blkid > b.blkid ) ? 1 : 0 );

Now TestSmallBlocks patch does not fail.


> Reducing memory consumption on Namenode : Part 1
> ------------------------------------------------
>
>                 Key: HADOOP-803
>                 URL: https://issues.apache.org/jira/browse/HADOOP-803
>             Project: Hadoop
>          Issue Type: Bug
>          Components: dfs
>            Reporter: Raghu Angadi
>         Assigned To: Raghu Angadi
>             Fix For: 0.11.0
>
>         Attachments: block-refs-2.patch, block-refs-3.patch, block-refs-5.patch, HADOOP-803.patch
>
>
> There appears to be some places in Namenode that allow reducing memory consumption without
intrusive code or feature changes. This bug is an initial attempt making those changes. Please
include your thoughts as well. 
> One change I am planning to make : 
> Currently one copy of each block exists for each of the replicas and one copy for blockMap.
I think they are all supposed to be same.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message