hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ivan Kelly (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-1580) Add interface for generic Write Ahead Logging mechanisms
Date Sun, 06 Mar 2011 10:45:45 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13003127#comment-13003127

Ivan Kelly commented on HDFS-1580:

To read in a log you pass in a URI to the getInputStream method. For BK, this will allow us
to have no explicit transfer stage. Files are a different matter. I haven't thought much about
it yet, but file URI do have a space for host. file://<host>/<path>. The FileJournalManager
could return a URI with the host part set for getLogs. This could be passed to the getInputStream
call, which would download it from the remote host if nescessary.

So, the steps would be as you outlined except for the "iterates over each of the RemoteLogs,
and fetches them". Instead you would just iterate over them and load them using the JournalManager.getInputStream
method. The implementation of the method would take care of whether the file is remote or

> Add interface for generic Write Ahead Logging mechanisms
> --------------------------------------------------------
>                 Key: HDFS-1580
>                 URL: https://issues.apache.org/jira/browse/HDFS-1580
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Ivan Kelly
>         Attachments: HDFS-1580+1521.diff, HDFS-1580.diff, generic_wal_iface.pdf, generic_wal_iface.pdf,

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message