hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-11187) Optimize disk access for last partial chunk checksum of Finalized replica
Date Fri, 28 Jul 2017 00:40:01 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16104214#comment-16104214
] 

Hadoop QA commented on HDFS-11187:
----------------------------------

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m  0s{color} | {color:blue}
Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red}  0m  4s{color} | {color:red}
HDFS-11187 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HDFS-11187 |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12840906/HDFS-11187.001.patch
|
| Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/20455/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT   http://yetus.apache.org |


This message was automatically generated.



> Optimize disk access for last partial chunk checksum of Finalized replica
> -------------------------------------------------------------------------
>
>                 Key: HDFS-11187
>                 URL: https://issues.apache.org/jira/browse/HDFS-11187
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: datanode
>            Reporter: Wei-Chiu Chuang
>            Assignee: Wei-Chiu Chuang
>         Attachments: HDFS-11187.001.patch
>
>
> The patch at HDFS-11160 ensures BlockSender reads the correct version of metafile when
there are concurrent writers.
> However, the implementation is not optimal, because it must always read the last partial
chunk checksum from disk while holding FsDatasetImpl lock for every reader. It is possible
to optimize this by keeping an up-to-date version of last partial checksum in-memory and reduce
disk access.
> I am separating the optimization into a new jira, because maintaining the state of in-memory
checksum requires a lot more work.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message