hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wei-Chiu Chuang (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-11187) Optimize disk access for last partial chunk checksum of Finalized replica
Date Sat, 03 Feb 2018 01:20:00 GMT

     [ https://issues.apache.org/jira/browse/HDFS-11187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Wei-Chiu Chuang updated HDFS-11187:
       Resolution: Fixed
    Fix Version/s: 3.1.0
           Status: Resolved  (was: Patch Available)

Thanks [~kihwal]!

Committed 005 patch to trunk.

> Optimize disk access for last partial chunk checksum of Finalized replica
> -------------------------------------------------------------------------
>                 Key: HDFS-11187
>                 URL: https://issues.apache.org/jira/browse/HDFS-11187
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: datanode
>            Reporter: Wei-Chiu Chuang
>            Assignee: Wei-Chiu Chuang
>            Priority: Major
>             Fix For: 3.1.0
>         Attachments: HDFS-11187.001.patch, HDFS-11187.002.patch, HDFS-11187.003.patch,
HDFS-11187.004.patch, HDFS-11187.005.patch
> The patch at HDFS-11160 ensures BlockSender reads the correct version of metafile when
there are concurrent writers.
> However, the implementation is not optimal, because it must always read the last partial
chunk checksum from disk while holding FsDatasetImpl lock for every reader. It is possible
to optimize this by keeping an up-to-date version of last partial checksum in-memory and reduce
disk access.
> I am separating the optimization into a new jira, because maintaining the state of in-memory
checksum requires a lot more work.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message