hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhe Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (HDFS-1499) mv the namenode NameSpace and BlocksMap to hbase to save the namenode memory
Date Fri, 28 Oct 2016 06:09:58 GMT

     [ https://issues.apache.org/jira/browse/HDFS-1499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Zhe Zhang resolved HDFS-1499.
    Resolution: Duplicate

Resolving the old JIRA since many similar JIRAs have been raised, including HDFS-8286.

> mv the namenode NameSpace and BlocksMap to hbase to save the namenode memory
> ----------------------------------------------------------------------------
>                 Key: HDFS-1499
>                 URL: https://issues.apache.org/jira/browse/HDFS-1499
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: namenode
>            Reporter: dl.brain.ln
> The NameNode stores all its metadata in the main memory of the machine on which it is
deployed. With the file-count and block number growing, namenode machine can't hold anymore
files and blocks in its memory and thus restrict the HDFS cluster growth. So many people are
talking and thinking abont this problem. Google's next version of GFS use bigtable to store
the metadata of the DFS and that seem works. What if we use hbase as the same?
> In the namenode structure, the namespace of the filesystem and the map of block ->
datanodes, datanode->blocks which keeped in memory are consume most of the namenode's heap,
what if we store those data structure in hbase to decrease the namenode's memory?

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message