hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Raghu Angadi (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-1149) DFS Scalability: high cpu usage in addStoredBlock
Date Wed, 28 Mar 2007 22:37:25 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-1149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12485015
] 

Raghu Angadi commented on HADOOP-1149:
--------------------------------------

Each addStoredBlock() invokes processOverReplicatedBlock(), which can be made a no-op in common
case easily. Currently it does some allocation and iterations. But it does not seem more than
rest of addStoredBlock(). Did you find processOverReplicatedBlock() in stack trace most of
the time when you observed this?

Let me know when you suspect this again, and we can check the trace. 


> DFS Scalability: high cpu usage in addStoredBlock
> -------------------------------------------------
>
>                 Key: HADOOP-1149
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1149
>             Project: Hadoop
>          Issue Type: Bug
>          Components: dfs
>            Reporter: dhruba borthakur
>         Assigned To: Raghu Angadi
>
> I have seen addStoredBlock() consume lots of CPU. One possible cause is that it invokes
proccessOverReplicatedBlock. The logic to find and purge over-replicated blocks can be done
much less often.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message