hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Luca Telloli (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-5188) Modifications to enable multiple types of logging
Date Wed, 20 May 2009 16:38:45 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-5188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Luca Telloli updated HADOOP-5188:

    Attachment: HADOOP-5188.patch

This patch adds the following features:
- supports for URI for property dfs.name.edits.dir (HADOOP-5832)
- abstraction of logging systems 

The abstraction might be still incomplete, but I think it's in a good shape. 

To achieve the first point (URI in dfs.name.edits.dir), the patch introduces a new abstraction
called LoggingDevice. 
The trunk implementation currently assumes that every attribute inside the property dfs.name.edits.dir
is a StorageDirectory, which is false if other types of logging are specified. LoggingDevice
should help with it; additionally, a Logging device would incorporate all needed parameters
for its instantiation by reading them from the initialization URI. 

Ideally, a new logging device should: 
- extend class LoggingDevice 
- implement interfaces EditLogInput/OutputStream

Additionally, in the current implementation there is one data structure that keeps the set
of edit streams (FSEditLog.editStreams) and one structure that keeps the list of StorageDirectories.
The two structures are maintained in sync whenever a edit stream fails (the corresponding
storage directory is removed) or a storage directory fails (the corresponding edit stream
is removed). 

My hope is that, by using logging devices, we could keep only one data structure. 

- the patch passes all NameNode unit tests
- I'm starting the submit patch process to get more feedback from users

> Modifications to enable multiple types of logging 
> --------------------------------------------------
>                 Key: HADOOP-5188
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5188
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: dfs
>    Affects Versions: 0.20.0
>            Reporter: Luca Telloli
>             Fix For: 0.21.0
>         Attachments: HADOOP-5188.patch, HADOOP-5188.patch, HADOOP-5188.patch, HADOOP-5188.patch,

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message