hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6581) Write to single replica in memory
Date Fri, 22 Aug 2014 02:01:17 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14106349#comment-14106349
] 

Colin Patrick McCabe commented on HDFS-6581:
--------------------------------------------

bq. If ramfs ate 90% of memory, would a YARN task die or instead just get their heaps swapped?

Depends on how much memory 10% of memory is, whether swap is enabled, etc. etc.

The point I was trying to make is that if 90% of ram goes missing, you will notice.  Whereas
if tmpfs starts swapping, you may not notice, unless you're paying careful attention to performance.
 tmpfs doesn't have the predictability that ramfs does.

> Write to single replica in memory
> ---------------------------------
>
>                 Key: HDFS-6581
>                 URL: https://issues.apache.org/jira/browse/HDFS-6581
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: datanode
>            Reporter: Arpit Agarwal
>            Assignee: Arpit Agarwal
>         Attachments: HDFSWriteableReplicasInMemory.pdf
>
>
> Per discussion with the community on HDFS-5851, we will implement writing to a single
replica in DN memory via DataTransferProtocol.
> This avoids some of the issues with short-circuit writes, which we can revisit at a later
time.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message