hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konstantin Shvachko (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-1188) processIOError() should update fstime file
Date Wed, 30 Jan 2008 01:07:34 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-1188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Konstantin Shvachko updated HADOOP-1188:
----------------------------------------

    Attachment: hadoop-1188.patch

This patch writes new time-stamp in fstime file if a directory becomes unavailable.
If writing of the time-stamp fails then this directory is going to be removed as well,
which makes checking of consistency of the edits files lengths on startup redundant.

Inequality of lengths of two edits files does necessarily mean inconsistency of the
storage directories. If one directory contains an image, which has already been
merged with the edits, and another has the image and the edits in the pre-merged
conditions, then the first edits file is empty, while the second is not, but the storage 
directories in fact define the same namespaces, that is they are equivalent.

> processIOError() should update fstime file
> ------------------------------------------
>
>                 Key: HADOOP-1188
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1188
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: dfs
>    Affects Versions: 0.10.1, 0.11.2, 0.12.2
>            Reporter: Konstantin Shvachko
>            Assignee: Konstantin Shvachko
>            Priority: Blocker
>             Fix For: 0.16.1
>
>         Attachments: hadoop-1188.patch
>
>
> Name-node can have multiple directories to store the name space image and edits.
> During startup the name-node selects the latest image and reads it in memory.
> fstime stores the time of the latest checkpoint.
> If one of the directories becomes inaccessible during normal operation the name-node
> excludes it from the list, and never writes anything into it until restarted.
> Now if the the name-node restarts after excluding one of the directories, and if that
> excluded directory contained the latest image, then all modifications of the name space
> recorded in edits files in other directories starting from the moment the directory was
> excluded will be lost.
> We should update fstime in all remaining good directories after the exclusion.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message