hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Boris Shkolnik (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-4045) Increment checkpoint if we see failures in rollEdits
Date Thu, 26 Mar 2009 19:06:50 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-4045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12689606#action_12689606
] 

Boris Shkolnik commented on HADOOP-4045:
----------------------------------------

Couple of words about the design. 
There are StorageDirectories(SDs) of different types: IMAGE or EDITS or both. Which means
some do not have any EditLogStreams associated with them. On the other hand there are some
EditLogStreams which are attached to SDs and some that are not (BACKUP node streaming). Thus
we should be able to take care of IOErrors from both sides. So the processIOError could be
called for an SD or for a EditLogStream (eStream). If it is called for a SD and this SD has
associated eStream we need to call processIOError for the stream too and vice-versa. So I
left one processIOError function in each class with an optional flag to specify if the error
should be propagated to the corresponding SD or eStream.
All functions accept arrayList as an argument.

> Increment checkpoint if we see failures in rollEdits
> ----------------------------------------------------
>
>                 Key: HADOOP-4045
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4045
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: dfs
>            Reporter: Lohit Vijayarenu
>            Assignee: Boris Shkolnik
>            Priority: Critical
>             Fix For: 0.19.2
>
>         Attachments: HADOOP-4045.patch
>
>
> In _FSEditLog::rollEdits_, if we encounter an error during opening edits.new, we remove
 the store directory associated with it. At this point we should also increment checkpoint
on all other directories.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message