hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hairong Kuang (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-988) Namenode should use single map for block to its meta data.
Date Wed, 21 Mar 2007 00:07:32 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12482616
] 

Hairong Kuang commented on HADOOP-988:
--------------------------------------

+1 The patch looks good. One comment is that the patch caculates the number of active blocks
to be equal to the blockMap size, which is not true all the time. Active blocks are the blocks
that belong to a file, but blockMap may contain blocks that do not belong to any file. Please
check if this causes any problem.

> Namenode should use single map for block to its meta data.
> ----------------------------------------------------------
>
>                 Key: HADOOP-988
>                 URL: https://issues.apache.org/jira/browse/HADOOP-988
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: dfs
>            Reporter: Raghu Angadi
>         Assigned To: Raghu Angadi
>         Attachments: HADOOP-988-1.patch
>
>
> This is a follow up from HADOOP-803. Currently there two maps which have similar functionality
:
>    1) blockMap : maps block to list of datanodes  that contain the block
>    2) activeBlocks : maps block to INode that it blongs to.
> Apart from simplifying, it saves 32 bytes per block and 24 bytes by avoid extra block
object we currently have for files that exist before Namenode starts (see HADOOP-803).
> We could combine these two into something like block to { containingNodes, INode, etc
}. 
> Another option is to get Move INode and list of dataNodes into Block object.
> Another option that requires bigger change is not use Block object but just 64 bit BlockId.
Then the map would be BlockId to all the block related info. I will file another Jira regd
not using Block object in NameNode.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message