hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ivan Kelly (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-2158) Add JournalSet to manage the set of journals.
Date Mon, 26 Sep 2011 09:46:26 GMT

    [ https://issues.apache.org/jira/browse/HDFS-2158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13114563#comment-13114563
] 

Ivan Kelly commented on HDFS-2158:
----------------------------------

Why not get rid of journal stream completely? Perhaps we can do this in a later JIRA.

Still have a findbug warning because of the custom synchronization in logSync.

     <!-- 
     synchronization is manually done in logSync()
     //-->
     <Match>
       <Class name="org.apache.hadoop.hdfs.server.namenode.FSEditLog" />
       <Field name="editLogStream" />
       <Bug pattern="IS2_INCONSISTENT_SYNC" />
     </Match>
     
 You've added a new log statement in EditLogFileOutputStream#flushAndSync. Shouldn't it be
a LOG.warn()? There should be any instance of calling flush without data having been written
to the stream first. The only case it could happen is when the stream has errored on a previous
write, but it should have been removed from JournalSetOutputStream at that point. In fact,
Im not sure how this worked in the old code either. 

In FSEditLog#close you removed the assert for !journals.isEmpty. Instead you could put assert
editLogStream != null;

In FSEditLog#logEdit, you catch the IOException, log and continue as normal. I can't see where
the error is handled later. I don't think editLogStream.setReadyToFlush() will throw an exception
if no active journals are found.

JournalSet#getSyncTimes doesn't return anything, so it's not really a getter. I think this
would be cleaner if getSyncTimes() just returned a String rather than taking in a StringBuilder.

{quote}
The finalizeLogSegments takes parameters which are not available in close method. 
{quote}
The last txid should be available to stream, as it must have passed through write(). Same
for first tx id. In any case, it's not that important to me, just something I thought would
make things nicer.


> Add JournalSet to manage the set of journals.
> ---------------------------------------------
>
>                 Key: HDFS-2158
>                 URL: https://issues.apache.org/jira/browse/HDFS-2158
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Jitendra Nath Pandey
>            Assignee: Jitendra Nath Pandey
>         Attachments: HDFS-2158.1.patch, HDFS-2158.3.patch, HDFS-2158.4.patch, HDFS-2158.8.patch,
HDFS-2158.9.patch
>
>
> The management of the collection of journals can be encapsulated in a JournalSet. This
will cleanup the FSEditLog code significantly.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message