hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-12678) Handle empty rename pending metadata file during atomic rename in redo path
Date Mon, 28 Dec 2015 15:28:49 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-12678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15072813#comment-15072813

Hadoop QA commented on HADOOP-12678:

| (x) *{color:red}-1 overall{color}* |
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 3s {color} | {color:red}
HADOOP-12678 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute
for help. {color} |
|| Subsystem || Report/Notes ||
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12779656/HADOOP-12678.001.patch
| JIRA Issue | HADOOP-12678 |
| Powered by | Apache Yetus 0.2.0-SNAPSHOT   http://yetus.apache.org |
| Console output | https://builds.apache.org/job/PreCommit-HADOOP-Build/8316/console |

This message was automatically generated.

> Handle empty rename pending metadata file during atomic rename in redo path
> ---------------------------------------------------------------------------
>                 Key: HADOOP-12678
>                 URL: https://issues.apache.org/jira/browse/HADOOP-12678
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs/azure
>            Reporter: madhumita chakraborty
>            Assignee: madhumita chakraborty
>            Priority: Critical
>         Attachments: HADOOP-12678.001.patch
> Handle empty rename pending metadata file during atomic rename in redo path
> During atomic rename we create metadata file for rename(-renamePending.json). We create
that in 2 steps
> 1. We create an empty blob corresponding to the .json file in its real location
> 2. We create a scratch file to which we write the contents of the rename pending which
is then copied over into the blob described in 1
> If process crash occurs after step 1 and before step 2 is complete - we will be left
with a zero size blob corresponding to the pending rename metadata file.
> This kind of scenario can happen in the /hbase/.tmp folder because it is considered a
candidate folder for atomic rename. Now when HMaster starts up it executes listStatus on the
.tmp folder to clean up pending data. At this stage due to the lazy pending rename complete
process we look for these json files. On seeing an empty file the process simply throws a
fatal exception assuming something went wrong.

This message was sent by Atlassian JIRA

View raw message