hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Cutting (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-1867) use single parameter to specify a node's available ram
Date Mon, 10 Sep 2007 18:53:29 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-1867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12526218

Doug Cutting commented on HADOOP-1867:

> Isn't memory allowed for JVM already available?

Good point.  So perhaps we should base things on Runtime#maxMemory()?  We could set the JVM
heap size based on /proc/meminfo, then base things on that.

Parameters whose values might be automatically determined include:
 - fs.inmemory.size.mb
 - io.sort.factor
 - io.sort.mb
 - io.file.buffer.size

What about mapred.child.java.opts?

> use single parameter to specify a node's available ram
> ------------------------------------------------------
>                 Key: HADOOP-1867
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1867
>             Project: Hadoop
>          Issue Type: Improvement
>            Reporter: Doug Cutting
> To simplify configuration, we should use a single parameter to indicate a node's available
RAM.  Sites should not need to adjust more than this single parameter to configure nodes available
memory.  In task JVMs, some significant percentage of the memory should be reserved for application
code, with the remainder divided among various system buffers.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message