hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Suresh Srinivas (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-4434) Provide a mapping from INodeId to INode
Date Mon, 08 Apr 2013 18:13:16 GMT

    [ https://issues.apache.org/jira/browse/HDFS-4434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13625615#comment-13625615
] 

Suresh Srinivas commented on HDFS-4434:
---------------------------------------

For a 1GB total java heapsize, the code attempts to allocated 32MB memory to GSet. However,
after running junit tests, the available memory is very low:
{noformat}
2013-04-06 00:44:04,716 INFO  util.GSet (LightWeightGSet.java:<init>(89)) - actual LightWeightGSet
size is 4194304 entries
2013-04-06 00:44:04,717 INFO  util.GSet (LightWeightGSet.java:<init>(90)) - maxMemory
1011286016
2013-04-06 00:44:04,717 INFO  util.GSet (LightWeightGSet.java:<init>(91)) - freeMemory
20897432
2013-04-06 00:44:04,717 INFO  util.GSet (LightWeightGSet.java:<init>(92)) - totalMemory
1011286016
{noformat}

It looks like some of the tests are no probably setting their fields to null. This results
in unused objects lingering on the heap and OOM.
                
> Provide a mapping from INodeId to INode
> ---------------------------------------
>
>                 Key: HDFS-4434
>                 URL: https://issues.apache.org/jira/browse/HDFS-4434
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: namenode
>    Affects Versions: 3.0.0
>            Reporter: Brandon Li
>            Assignee: Suresh Srinivas
>         Attachments: HDFS-4434.patch, HDFS-4434.patch, HDFS-4434.patch, HDFS-4434.patch,
HDFS-4434.patch, HDFS-4434.patch, HDFS-4434.patch, HDFS-4434.patch, HDFS-4434.patch, HDFS-4434.patch,
HDFS-4434.patch, HDFS-4434.patch, HDFS-4434.patch, HDFS-4434.patch
>
>
> This JIRA is to provide a way to access the INode via its id. The proposed solution is
to have an in-memory mapping from INodeId to INode. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message