hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ming Ma (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-7587) Edit log corruption can happen if append fails with a quota violation
Date Fri, 31 Jul 2015 20:43:05 GMT

     [ https://issues.apache.org/jira/browse/HDFS-7587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Ming Ma updated HDFS-7587:
--------------------------
    Attachment: HDFS-7587-branch-2.6.patch

For the 2.6.1 release effort, the backport isn't straightforward due to difference between
2.6 and 2.7. It has the following differences compared to the original patch.

* Include part of HDFS-7509 so that prepareFileForWrite has the expected function signature.
* Use Quota.Counts instead of QuotaCounts which is introduced in HDFS-7584.
* Skip the check for storage type specific quota introduced in HDFS-7584.
* Add the necessary definitions for INodesPath#length and FSDirectory#shouldSkipQuotaChecks.

> Edit log corruption can happen if append fails with a quota violation
> ---------------------------------------------------------------------
>
>                 Key: HDFS-7587
>                 URL: https://issues.apache.org/jira/browse/HDFS-7587
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: namenode
>            Reporter: Kihwal Lee
>            Assignee: Jing Zhao
>            Priority: Blocker
>              Labels: 2.6.1-candidate
>             Fix For: 2.7.0
>
>         Attachments: HDFS-7587-branch-2.6.patch, HDFS-7587.001.patch, HDFS-7587.002.patch,
HDFS-7587.003.patch, HDFS-7587.patch
>
>
> We have seen a standby namenode crashing due to edit log corruption. It was complaining
that {{OP_CLOSE}} cannot be applied because the file is not under-construction.
> When a client was trying to append to the file, the remaining space quota was very small.
This caused a failure in {{prepareFileForWrite()}}, but after the inode was already converted
for writing and a lease added. Since these were not undone when the quota violation was detected,
the file was left in under-construction with an active lease without edit logging {{OP_ADD}}.
> A subsequent {{append()}} eventually caused a lease recovery after the soft limit period.
This resulted in {{commitBlockSynchronization()}}, which closed the file with {{OP_CLOSE}}
being logged.  Since there was no corresponding {{OP_ADD}}, edit replaying could not apply
this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message