hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Raghu Angadi (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-988) Namenode should use single map for block to its meta data.
Date Fri, 16 Mar 2007 19:12:09 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Raghu Angadi updated HADOOP-988:

    Attachment: HADOOP-988-1.patch

Attached 1.patch for review. I have done fair amount of testing.

> For each block : + BlockInfo object (24 bytes) - HashMap$Entry(24bytes in activeBlocks)
- Block (24 ... 

This becomes : + BlockInfo (24) - HashMap$Entry(24, in activeBlocks)  - (datanode) ArrayList
(24) - Block(24 for block existing before namenode restarts.
i.e. 48 byte per  block.

Since this patch touches various places in FSNamesystem.java, I also did minor code changes
and removed some allocations here and there (e.g. in open()).

> Namenode should use single map for block to its meta data.
> ----------------------------------------------------------
>                 Key: HADOOP-988
>                 URL: https://issues.apache.org/jira/browse/HADOOP-988
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: dfs
>            Reporter: Raghu Angadi
>         Assigned To: Raghu Angadi
>         Attachments: HADOOP-988-1.patch
> This is a follow up from HADOOP-803. Currently there two maps which have similar functionality
>    1) blockMap : maps block to list of datanodes  that contain the block
>    2) activeBlocks : maps block to INode that it blongs to.
> Apart from simplifying, it saves 32 bytes per block and 24 bytes by avoid extra block
object we currently have for files that exist before Namenode starts (see HADOOP-803).
> We could combine these two into something like block to { containingNodes, INode, etc
> Another option is to get Move INode and list of dataNodes into Block object.
> Another option that requires bigger change is not use Block object but just 64 bit BlockId.
Then the map would be BlockId to all the block related info. I will file another Jira regd
not using Block object in NameNode.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message