hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "dhruba borthakur (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-2340) Limiting memory usage on namenode
Date Mon, 03 Dec 2007 23:55:43 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-2340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12548056
] 

dhruba borthakur commented on HADOOP-2340:
------------------------------------------

One line of reasoning is that if we never timeout client RPC requests (HADOOP-2188), then
the above situation will not occur. A GC run on the Namenode will cause clients to block and
slowdown. My feeling is that we should observe the system post-2188 and then decide whether
(and policy) we need to monitor Namenode resources.



> Limiting memory usage on namenode
> ---------------------------------
>
>                 Key: HADOOP-2340
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2340
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: dfs
>            Reporter: dhruba borthakur
>
> When memory usage is too high, the namenode could refuse creation of new files.  This
would still crash applications, but it would keep the filesystem itself from crashing in a
way that is hard to recover while folks remove excessive (presumably) small files.
> http://java.sun.com/javase/6/docs/api/java/lang/management/MemoryPoolMXBean.html#UsageThreshold
> Other resource limitations (other than memory) are CPU, network and disk. We thought
that we do not need to monitor those resources. The monitoring of critical resources and the
policy of what action to take can be outside the actual Namenode process itself.
> There are two reasons that cause memory pressure on the Namenode. One is the creation
of a large number of files. This reduces the free memory pool and the GC has to work even
harder to recycle memory. The other reason is when a burst of RPCs arrive at the Namenode
(especially Block reports). This spurt causes free memory to reduce dramatically within a
couple of seconds and makes GC work harder. And we know that when GC runs hard, the server
threads in the JVM starve for CPU, causing timeouts on clients.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message