hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Luca Telloli (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-6001) Incorporate storage directories into EditLogFileInput/Output streams
Date Fri, 19 Jun 2009 17:20:07 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-6001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12721887#action_12721887

Luca Telloli commented on HADOOP-6001:

I guess we need some additional clarification on the purpose of this patch and on the interrelationship
between this one and the other ones we're working on (5188, 5189, 5832, etc). 

The overall goal of 5188 is simple: to have the EditLogInput/OuputStream to properly incapsulate
everything is needed for log of NameNode operations, and FSEditLog to call the same methods
independently of the type of stream it's working on. 

Then, with 5832 I moved a step towards this goal: to treat the properties in the .xml file
which specify the log devices and their parameters as a URI. While deploying that patch I
realized that I couldn't pass a URI as argument to EditLogFileInput/Output Stream, because
the constructor was dependent on the role of the file. By role of the file, I mean edits or
edits.new, two different files used in file-based logging to take care of changes. You can
see that if you look at the original code for FSEditLog, when EditLogFileOutputStream is instantiated
at times with the EDITS file, and some other times with the EDITS_NEW file. This is bad, because
it breaks generality. 

With some simple modifications, contained in this patch, I can now pass to EditLogFileOutputStream
the container of these files, which is a StorageDirectory and, depending on the call from
FSEditLog, I can switch between edits and edits.new inside EditLogFileOutputStream, making
it transparent to FSEditLog which doesn't have to instantiate this class based on a specific
file and its role. 
At the same time, a StorageDirectory is exactly what is passed inside the URI of a file-based
logger, which connects this patch to 5832. 

I hope this clarifies some of the doubt on this patch, but I'm happy to answer more questions
if they come up. 

> Incorporate storage directories into EditLogFileInput/Output streams
> --------------------------------------------------------------------
>                 Key: HADOOP-6001
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6001
>             Project: Hadoop Core
>          Issue Type: Sub-task
>            Reporter: Luca Telloli
>         Attachments: HADOOP-6001.patch, HADOOP-6001.patch

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message