hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-2187) HDFS-1580: Make EditLogInputStream act like an iterator over FSEditLogOps
Date Thu, 04 Aug 2011 02:01:28 GMT

     [ https://issues.apache.org/jira/browse/HDFS-2187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Todd Lipcon updated HDFS-2187:

    Attachment: hdfs-2187.txt

Rather than write out separate comments, I just took Ivan's patch and ran with it a bit. Here
are the changes I'm suggesting in the form of a diff:
- make EditLogInputStream implement Closeable, so we can still use IOUtils.closeStream on
- make EditLogBackupInputStream.readOp() throw IllegalStateException instead of IOE if readOp
is called without setting data first -- this would be a coding error rather than something
we expect to see
- remove some redundant creation of new DataInputStreams
- remove unused imports in EditLogFileInputStream
- make EditLogFileInputStream members final
- adds a new exception type in EditLogFileInputStream's constructor, so it's possible to differentiate
between inability to open a file and an invalid header. The earlier iteration of the patch
would end up storing logVersion as 0, and failing an assertion, I think.
- made the validateEditLog call that takes a stream _not_ close the stream when it finishes
validation -- generally I think it's best that whoever opens a stream be the one that closes
it. Added a wrapper function that takes a File, and handles opening and closing the stream

Are you guys OK with these changes? I attached both the delta and the new combined patch

> HDFS-1580: Make EditLogInputStream act like an iterator over FSEditLogOps
> -------------------------------------------------------------------------
>                 Key: HDFS-2187
>                 URL: https://issues.apache.org/jira/browse/HDFS-2187
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Ivan Kelly
>            Assignee: Ivan Kelly
>             Fix For: 0.23.0
>         Attachments: 2187-changes.txt, HDFS-2187.diff, HDFS-2187.diff, hdfs-2187.txt
> This JIRA is for the input side changes moved out of HDFS-2149. EditLogInputStream has
been changed to no longer be an InputStream implementation, but to return a stream of FSEditLogOp
objects using readOp(). The upshot is that all that can ever be read from an EditLogInputStream
is a op. No random hackery can be used to put other things in the stream. Version is now a
property of the EditLogInputStream and retrieved using getVersion().

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message