hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "dhruba borthakur (JIRA)" <j...@apache.org>
Subject [jira] Created: (HADOOP-1117) DFS Scalability: When the namenode is restarted it consumes 80% CPU
Date Tue, 13 Mar 2007 22:54:09 GMT
DFS Scalability: When the namenode is restarted it consumes 80% CPU
-------------------------------------------------------------------

                 Key: HADOOP-1117
                 URL: https://issues.apache.org/jira/browse/HADOOP-1117
             Project: Hadoop
          Issue Type: Bug
          Components: dfs
    Affects Versions: 0.12.0
            Reporter: dhruba borthakur
         Assigned To: dhruba borthakur


When the namenode is restarted, the datanodes register and each block is inserted into neededReplication.
When the namenode exists, safemode it sees starts processing neededReplication. It picks up
a block from neededReplication, sees that it has already has the required number of replicas,
and continues to the next block in neededReplication. The blocks remain in neededReplication
permanentlyhe namenode worker thread to scans this huge list of blocks once every 3 seconds.
This consumes plenty of CPU on the namenode.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message