hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arpit Agarwal (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6919) Enforce a single limit for RAM disk usage and replicas cached via locking
Date Fri, 10 Apr 2015 20:48:14 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14490336#comment-14490336
] 

Arpit Agarwal commented on HDFS-6919:
-------------------------------------

As a first step the memory writer can reserve space for a full replica in FsDatasetCache.
The memory will be released if a block is finalized at less than full length or when the block
is evicted from ramfs. This gets us towards a unified configuration for both features via
{{dfs.datanode.max.locked.memory}}. I looked at the FsDatasetCache code and the change seems
easy in theory.

Next we could add a hook for CCM to evict ramfs replicas.

> Enforce a single limit for RAM disk usage and replicas cached via locking
> -------------------------------------------------------------------------
>
>                 Key: HDFS-6919
>                 URL: https://issues.apache.org/jira/browse/HDFS-6919
>             Project: Hadoop HDFS
>          Issue Type: Bug
>            Reporter: Arpit Agarwal
>            Assignee: Arpit Agarwal
>
> The DataNode can have a single limit for memory usage which applies to both replicas
cached via CCM and replicas on RAM disk.
> See comments [1|https://issues.apache.org/jira/browse/HDFS-6581?focusedCommentId=14106025&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14106025],
[2|https://issues.apache.org/jira/browse/HDFS-6581?focusedCommentId=14106245&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14106245]
and [3|https://issues.apache.org/jira/browse/HDFS-6581?focusedCommentId=14106575&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14106575]
for discussion.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message