hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rakesh R (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-14402) Improve the implementation for HDFS cache on SCM
Date Tue, 14 May 2019 02:42:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-14402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16839033#comment-16839033

Rakesh R commented on HDFS-14402:

Thanks [~PhiloHe] for taking this jira ahead. Please go through below review comments on
the patch:
# Checksum verification uses {{blockChannel}} and reads content at the replica device throughput.
Again, reading block content make the replica device busy and will affect other client read/write
operation, assume replica device is a HDD. Since the data is written to NVMe, you can read
it back from NVMe. Also, this will act as a verification checkpoint to ensure there is no
data loss after writing the block to NVMe device, right?
	verifyChecksum(length, metaIn, blockChannel, blockFileName);
# Can we make checksum compute on/off based on some configuration and this can improve performance
and not make the device busy. We can introduce this tuning on/off parameter applicable to
all MappableBlockLoaders.
# Please change {{protected int fillBuffer}} to {{private}} visibility.

> Improve the implementation for HDFS cache on SCM
> ------------------------------------------------
>                 Key: HDFS-14402
>                 URL: https://issues.apache.org/jira/browse/HDFS-14402
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: caching, datanode
>            Reporter: Feilong He
>            Assignee: Feilong He
>            Priority: Major
>         Attachments: HDFS-14402.000.patch, HFDS-14402.001.patch
> We will consider to use transferTo API to improve SCM's cach performace.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message