hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wei-Chiu Chuang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-14581) EC: Append validation should be done before getting lease on file
Date Wed, 19 Jun 2019 23:19:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-14581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16868093#comment-16868093
] 

Wei-Chiu Chuang commented on HDFS-14581:
----------------------------------------

[~surendrasingh] thanks for reporting the issue.
Would you mind updating the jira summary? It looks too harmless. It should be like "Appending
to EC files crashes NameNode" and then folks will then jump on to this one.

I haven't reviewed this carefully, but I would hope to make the exception message more actionable.
An application developer wouldn't know how to deal with "Append on EC file without new block
is not supported." Thoughts? [~ayushtkn]

> EC: Append validation should be done before getting lease on file
> -----------------------------------------------------------------
>
>                 Key: HDFS-14581
>                 URL: https://issues.apache.org/jira/browse/HDFS-14581
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: erasure-coding
>    Affects Versions: 3.3.0
>            Reporter: Surendra Singh Lilhore
>            Assignee: Surendra Singh Lilhore
>            Priority: Critical
>         Attachments: HDFS-14581.001.patch
>
>
> *org.apache.hadoop.hdfs.server.namenode.FSDirAppendOp.prepareFileForAppend(..)@189*
> {noformat}
> file.recordModification(iip.getLatestSnapshotId());
> file.toUnderConstruction(leaseHolder, clientMachine);
> fsn.getLeaseManager().addLease(
> file.getFileUnderConstructionFeature().getClientName(), file.getId());
> LocatedBlock ret = null;
> if (!newBlock) {
> if (file.isStriped()) {
> throw new UnsupportedOperationException(
> "Append on EC file without new block is not supported.");
> }{noformat}
> In this code "UnsupportedOperationException" exception thows after marking file underConstruction.
In this case file is opened without any "Open" editlogs, after some time lease manager close
this file and add close edit log.
> When SBN tail this edit log, it will fail with this exception.
> {noformat}
> 2019-06-13 19:17:51,513 ERROR org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader:
Encountered exception on operation CloseOp [length=0, inodeId=0, path=/ECtest/********, replication=1,
mtime=1560261947480, atime=1560258249117, blockSize=134217728, blocks=[blk_-9223372036854775792_1005],
permissions=root:hadoop:rw-r--r--, aclEntries=null, clientName=, clientMachine=, overwrite=false,
storagePolicyId=0, erasureCodingPolicyId=0, opCode=OP_CLOSE, txid=1363]
> java.io.IOException: File is not under construction: /ECtest/container-executor
> at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.applyEditLogOp(FSEditLogLoader.java:504)
> at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:286)
> at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:181)
> at org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:924)
> at org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer.doTailEdits(EditLogTailer.java:329)
> at org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.doWork(EditLogTailer.java:460)
> at org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.access$300(EditLogTailer.java:410)
> at org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread$1.run(EditLogTailer.java:427)
> at org.apache.hadoop.security.SecurityUtil.doAsLoginUserOrFatal(SecurityUtil.java:485)
> at org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.run(EditLogTailer.java:423)
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message