hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsz Wo Nicholas Sze (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6934) Move checksum computation off the hot path when writing to RAM disk
Date Sat, 04 Oct 2014 02:31:33 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14158900#comment-14158900
] 

Tsz Wo Nicholas Sze commented on HDFS-6934:
-------------------------------------------

Thanks Arpit for the comment.  The CPU usage in DN won't be increased:
- For single replica case, if client sends checksum, then DN will verify it (i.e. DN needs
to compute checksum).  If client does not send checksum, DN will compute it when persisting
the replica.
- For multiple replica case, client must send checksum since only the first DN could write
to memory.  The other DNs still need checksum.

> Move checksum computation off the hot path when writing to RAM disk
> -------------------------------------------------------------------
>
>                 Key: HDFS-6934
>                 URL: https://issues.apache.org/jira/browse/HDFS-6934
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: datanode
>            Reporter: Arpit Agarwal
>            Assignee: Tsz Wo Nicholas Sze
>
> Since local RAM is considered reliable we can avoid writing checksums on the hot path
when replicas are being written to a local RAM disk.
> The checksum can be computed by the lazy writer when moving replicas to disk.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message