hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gabor Bota (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-11187) Optimize disk access for last partial chunk checksum of Finalized replica
Date Tue, 20 Feb 2018 18:35:02 GMT

     [ https://issues.apache.org/jira/browse/HDFS-11187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Gabor Bota updated HDFS-11187:
------------------------------
    Status: Patch Available  (was: Reopened)

Patch submitted for branch-2.7.
Cherry-picking the commit from branch-2.
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/FinalizedReplica.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java


> Optimize disk access for last partial chunk checksum of Finalized replica
> -------------------------------------------------------------------------
>
>                 Key: HDFS-11187
>                 URL: https://issues.apache.org/jira/browse/HDFS-11187
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: datanode
>            Reporter: Wei-Chiu Chuang
>            Assignee: Gabor Bota
>            Priority: Major
>             Fix For: 3.1.0, 2.10.0, 2.9.1, 2.8.4, 3.0.2
>
>         Attachments: HDFS-11187-branch-2.001.patch, HDFS-11187-branch-2.002.patch, HDFS-11187-branch-2.003.patch,
HDFS-11187-branch-2.004.patch, HDFS-11187-branch-2.7.001.patch, HDFS-11187.001.patch, HDFS-11187.002.patch,
HDFS-11187.003.patch, HDFS-11187.004.patch, HDFS-11187.005.patch
>
>
> The patch at HDFS-11160 ensures BlockSender reads the correct version of metafile when
there are concurrent writers.
> However, the implementation is not optimal, because it must always read the last partial
chunk checksum from disk while holding FsDatasetImpl lock for every reader. It is possible
to optimize this by keeping an up-to-date version of last partial checksum in-memory and reduce
disk access.
> I am separating the optimization into a new jira, because maintaining the state of in-memory
checksum requires a lot more work.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message