hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Luca Telloli (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HDFS-311) Modifications to enable multiple types of logging
Date Fri, 10 Jul 2009 16:41:14 GMT

     [ https://issues.apache.org/jira/browse/HDFS-311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Luca Telloli updated HDFS-311:
------------------------------

    Attachment: HDFS-311.patch

Usage: 
- This patch requires HDFS-397 to be applied first
- To use bookkeeper, you should specify the same values in dfs.name.edits.dir and fs.checkpoint.name.edits

- The value should be specified as a URI: for instance: bookkeeper://127.0.0.1:2181?nbookies=3&amp;quorumsize=2&amp;mode=verifiable
; at the moment, the three parameters are the number of bookies (nbookies), the size of the
quorum (quorumsize) and the quorum mode (mode). More informations on BookKeeper are available
at the Zookeeper website

Drawbacks: 
- The patch does not work with BackupNode yet, it works only with SecondaryNamenode 
- The patch passes all unit tests excluding TestDFSUpgrade and TestDistributedUpgrade. Since
these two tests rely on an upgrade scenario, I decided to post the patch anyway
- The patch includes old HADOOP-5189 (now HDFS-234)

Short list of modifications: 
EditLogOutputStream: 
- two abstract methods have been added 
	-  abstract void setLatestCheckpointTime(long time) throws IOException;  
	-  abstract void format(FSImage fsimage) throws IOException;

EditLogInputStream: 
- one abstract method has been added: abstract long getLatestCheckpointTime() throws IOException;

Journals: is a map <URI, JournalType>, used to retrieve all Journal of some specific
type. It's similar to StorageDirectory in that it provides a custom iterator  

FSImage: 
- readCheckpointTime and writeCheckPoint time have been made static. This should be thread
safe and allows EditLogOutputStream to access these methods 
- processIOError(ArrayList<StorageDirectory> sds, boolean propagate)  has been changed
with processIOError(List<URI> faultyJournals, boolean propagate)

FSEditLog:
processIOError(ArrayList<StorageDirectory> sds, boolean propagate)  has been changed
with processIOError(List<URI> faultyJournals, boolean propagate)

> Modifications to enable multiple types of logging 
> --------------------------------------------------
>
>                 Key: HDFS-311
>                 URL: https://issues.apache.org/jira/browse/HDFS-311
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Luca Telloli
>            Assignee: Luca Telloli
>         Attachments: HADOOP-5188.patch, HADOOP-5188.patch, HADOOP-5188.patch, HADOOP-5188.patch,
HADOOP-5188.patch, HADOOP-5188.patch, HADOOP-5188.pdf, HDFS-311.patch
>
>


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message