hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ivan Kelly (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-1893) Change edit logs and images to be named based on txid
Date Thu, 12 May 2011 17:39:47 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13032533#comment-13032533

Ivan Kelly commented on HDFS-1893:

Mostly looks good. There's a couple of TODOs and things which will need to be addressed, but
I guess there's be a polishing JIRA before any merge back into trunk.

I don't like the call to finalizeLogSegment from JournalAndStream. A member variable is being
maintained segmentStartsAtTxId, which would be better incapsulated inside the JournalAndStream
or even the EditLogOutputStream itself, as it is a property of the segment, not that which
is writing to it. I understand the rational for keeping this code out of EditLogOutputStream,
but I don't understand why this needs to be called from JournalAndStream. I think it would
be better for the stream to notify it's manager whenever it is closed. This way the segment
is _always_ finalised on a close. 

So, I would propose the following.
public class EditLogFileOutputStream {
   public interface ClosureListener {
      public void streamClosed();
   private final ClosureListener listener;

   public EditLogFileOutputStream(File name, int size, ClosureListener listener);

   public void close() {

public FileJournalManager implements JournalManager, EditLogFileOutputStream.ClosureListener
   // etc, etc

   EditLogOutputStream startLogSegment(long txid) {
      return new EditLogFileOutputStream(file, sizeOutputFlushBuffer, this);

   void streamClosed() {
      // what finalize current does.
This removes the need for FSEditLog to know anything about the lifecycle of the streams. It
currently has to know that finalizeLogSegment has to be called after stream close, which is

> Change edit logs and images to be named based on txid
> -----------------------------------------------------
>                 Key: HDFS-1893
>                 URL: https://issues.apache.org/jira/browse/HDFS-1893
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: name-node
>    Affects Versions: Edit log branch (HDFS-1073)
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>             Fix For: Edit log branch (HDFS-1073)
>         Attachments: hdfs-1893-prelim.txt
> This is the main subtask of HDFS-1073: actually switch over the naming of the files to
the new format as described in the design doc.
> I imagine it will be split out into a couple separate JIRAs before being committed, but
this still be the "big kahuna" patch.

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message