hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wei-Chiu Chuang (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-11187) Optimize last partial chunk checksum calculation for Finalized replica
Date Tue, 29 Nov 2016 18:26:58 GMT
Wei-Chiu Chuang created HDFS-11187:
--------------------------------------

             Summary: Optimize last partial chunk checksum calculation for Finalized replica
                 Key: HDFS-11187
                 URL: https://issues.apache.org/jira/browse/HDFS-11187
             Project: Hadoop HDFS
          Issue Type: Improvement
          Components: datanode
            Reporter: Wei-Chiu Chuang
            Assignee: Wei-Chiu Chuang


The patch at HDFS-11160 ensures BlockSender reads the correct version of metafile when there
are concurrent writers.

However, the implementation is not optimal, because it must always read the last partial chunk
checksum from disk while holding FsDatasetImpl lock for every reader. It is possible to optimize
this by keeping an up-to-date version of last partial checksum in-memory and reduce disk access.

I am separating the optimization into a new jira, because maintaining the state of in-memory
checksum requires a lot more work.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org


Mime
View raw message