hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ivan Kelly (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-1073) Simpler model for Namenode's fs Image and edit Logs
Date Fri, 15 Jul 2011 09:23:03 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13065801#comment-13065801

Ivan Kelly commented on HDFS-1073:

LogHeader has a read method but not a write. Will it make sense to encapsulate both read and
write of the header in the same class?
Agreed - Ivan has opened HDFS-2149 - I'd propose we do that under that JIRA?
HDFS-2149 will probably remove LogHeader completely. I plan to add a getVersion() call to
InputStreams and each stream will handle it's own metadata internally. So EditLogFileInputStream
will read it's version on creation, or first call to read etc. The input and output stream
will be packet based, so an input stream is basically an iterator over FSEditLogOp objects
and output stream is a sink for FSEditLogOp objects. I think the way I've implemented the
FSEditLogOp objects should avoid all extra copies and object creation. Whats more, there's
plenty to room to improve this by removing the creation of ArrayWritables and DeprecatedUTF8
objects and just write strings and arrays directly. 

> Simpler model for Namenode's fs Image and edit Logs 
> ----------------------------------------------------
>                 Key: HDFS-1073
>                 URL: https://issues.apache.org/jira/browse/HDFS-1073
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Sanjay Radia
>            Assignee: Todd Lipcon
>         Attachments: hdfs-1073-editloading-algos.txt, hdfs-1073.txt, hdfs1073.pdf, hdfs1073.pdf,
hdfs1073.pdf, hdfs1073.tex
> The naming and handling of  NN's fsImage and edit logs can be significantly improved
resulting simpler and more robust code.

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message