hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Raghu Angadi (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-764) The memory consumption of processReport() in the namenode can be reduced
Date Thu, 07 Dec 2006 18:34:22 GMT
    [ http://issues.apache.org/jira/browse/HADOOP-764?page=comments#action_12456515 ] 
            
Raghu Angadi commented on HADOOP-764:
-------------------------------------

Regd  references:

Say Bx is a block in Namenode.blockMap and Bx_ref is in Dx.blocks as desired. Now when we
update Dx.block in processReport(), we do Dx.blocks.clear() : i.e. Bx_ref is removed. Bx_new_ref
in newReport array is added to Dx.block. So there are two separate objects referenced by namenode.blockMap
and Dx.block. 

newReport[] does not hold reference to blocks in namenode.blockMap (as far as I can see).
What I am missing?




> The memory consumption of processReport() in the namenode can be reduced
> ------------------------------------------------------------------------
>
>                 Key: HADOOP-764
>                 URL: http://issues.apache.org/jira/browse/HADOOP-764
>             Project: Hadoop
>          Issue Type: Bug
>          Components: dfs
>            Reporter: dhruba borthakur
>         Assigned To: dhruba borthakur
>         Attachments: processBlockReport3.patch
>
>
> The FSNamesystem.processReport() method converts the blocklist for a datanode into an
array by calling node.getBlocks(). Although this memory allocation is transient, it could
possibly require the garbage-collector to work that much harder. 
> The method Block.getBlocks() should be deprecated. Code that currently uses this method
should instead iterate over the Collection.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message