hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhe Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-8996) Consolidate validateLog and scanLog in FJM#EditLogFile
Date Thu, 10 Sep 2015 22:24:45 GMT

     [ https://issues.apache.org/jira/browse/HDFS-8996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Zhe Zhang updated HDFS-8996:
----------------------------
    Attachment: HDFS-8996.00.patch

{{validateLog}} and {{scanLog}} have very similar logics in {{FileJournalManager}}, {{EditLogFileInputStream}},
{{FSEditLogLoader}}, and {{FSEditLogOp}}. The only difference is that when reading a future
layout version, the _scan_ logic only tries to parse the "frame" of the op, and is therefore
faster.

I wonder if we should eliminate {{validateLog}} altogether, since {{scanLog}} is already validating
checksums, after HDFS-8965.

Uploading initial patch to demonstrate the idea and trigger Jenkins. Will finish the refactor
(remove {{validateLog}} in all places) in the next rev.

> Consolidate validateLog and scanLog in FJM#EditLogFile
> ------------------------------------------------------
>
>                 Key: HDFS-8996
>                 URL: https://issues.apache.org/jira/browse/HDFS-8996
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: journal-node, namenode
>    Affects Versions: 2.0.0-alpha
>            Reporter: Zhe Zhang
>            Assignee: Zhe Zhang
>         Attachments: HDFS-8996.00.patch
>
>
> After HDFS-8965 is committed, {{scanEditLog}} will be identical to {{validateEditLog}}
in {{EditLogInputStream}} and {{FSEditlogLoader}}. This is a place holder for us to remove
the redundant {{scanEditLog}} code.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message