hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-2088) Move edits log archiving logic into FSEditLog/JournalManager
Date Mon, 20 Jun 2011 22:59:47 GMT

     [ https://issues.apache.org/jira/browse/HDFS-2088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Todd Lipcon updated HDFS-2088:

    Attachment: hdfs-2088.txt

Patch does the following:
- once StorageArchivalManager determines the minimum txid that needs to be retained, it simply
passes it along to FSEditLog.archiveLogsOlderThan.
- FSEditLog now propagates this through to all of the journal managers
- refactors some code in FSImageTransactionalStorageInspector into a static method {{matchEditLogs}}
so that FileJournalManager can share it. This will eventually move into FileJournalManager
itself like Ivan did in HDFS-2018, once the load-time stuff gets split up.
- adds a functional test to show that edits logs keep getting archived in an edits directory
even if it's considered "failed" as an image directory

> Move edits log archiving logic into FSEditLog/JournalManager
> ------------------------------------------------------------
>                 Key: HDFS-2088
>                 URL: https://issues.apache.org/jira/browse/HDFS-2088
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: name-node
>    Affects Versions: Edit log branch (HDFS-1073)
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>             Fix For: Edit log branch (HDFS-1073)
>         Attachments: hdfs-2088.txt
> Currently the logic to archive edits logs is File-specific which presents some issues
for Ivan's work. Since it relies on inspecting storage directories using NNStorage.inspectStorageDirs,
it also misses directories that the image layer considers "failed" which results in edits
logs piling up in these kinds of directories. This JIRA is similar to HDFS-2018 but only deals
with archival for now.

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message