hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lukas Majercak (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-11713) Use MoveFileEx to allow renaming a file when the destination exists
Date Thu, 27 Apr 2017 23:00:09 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15987846#comment-15987846
] 

Lukas Majercak commented on HDFS-11713:
---------------------------------------

Exception in {{TestRollingUpgrade.testRollback}} :

{code}
from E:\workspace\Hadoop\hadoop-hdfs-project\hadoop-hdfs\target\test\data\dfs\data\data1\current\BP-107441638-10.123.116.86-1493168359567\tmp\subdir0\subdir0\blk_1073741826_1002.meta
to E:\workspace\Hadoop\hadoop-hdfs-project\hadoop-hdfs\target\test\data\dfs\data\data1\current\BP-107441638-10.123.116.86-1493168359567\current\finalized\subdir0\subdir0\blk_1073741826_1002.meta
    at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.moveBlockFiles(FsDatasetImpl.java:882)
    at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.BlockPoolSlice.addBlock(BlockPoolSlice.java:286)
    at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsVolumeImpl.addFinalizedBlock(FsVolumeImpl.java:799)
    at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.finalizeReplica(FsDatasetImpl.java:1621)
    at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.updateReplicaUnderRecovery(FsDatasetImpl.java:2526)
    at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.updateReplicaUnderRecovery(FsDatasetImpl.java:2451)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.updateReplicaUnderRecovery(DataNode.java:2633)
    at org.apache.hadoop.hdfs.server.datanode.BlockRecoveryWorker$BlockRecord.updateReplicaUnderRecovery(BlockRecoveryWorker.java:87)
    at org.apache.hadoop.hdfs.server.datanode.BlockRecoveryWorker$BlockRecord.access$500(BlockRecoveryWorker.java:70)
    at org.apache.hadoop.hdfs.server.datanode.BlockRecoveryWorker$RecoveryTaskContiguous.syncBlock(BlockRecoveryWorker.java:241)
    at org.apache.hadoop.hdfs.server.datanode.BlockRecoveryWorker$RecoveryTaskContiguous.recover(BlockRecoveryWorker.java:153)
    at org.apache.hadoop.hdfs.server.datanode.BlockRecoveryWorker$1.run(BlockRecoveryWorker.java:541)
    at java.lang.Thread.run(Thread.java:745)
Caused by: 183: Cannot create a file when that file already exists.
    at org.apache.hadoop.io.nativeio.NativeIO.renameTo0(Native Method)
    at org.apache.hadoop.io.nativeio.NativeIO.renameTo(NativeIO.java:902)
    at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.moveBlockFiles(FsDatasetImpl.java:880)
    ... 12 more
{code}

> Use MoveFileEx to allow renaming a file when the destination exists
> -------------------------------------------------------------------
>
>                 Key: HDFS-11713
>                 URL: https://issues.apache.org/jira/browse/HDFS-11713
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: hdfs, native, rolling upgrades
>    Affects Versions: 2.7.1, 2.7.3, 3.0.0-alpha1, 3.0.0-alpha2
>            Reporter: Lukas Majercak
>            Assignee: Lukas Majercak
>              Labels: windows
>             Fix For: 3.0.0-alpha2
>
>         Attachments: HDFS-11713.001.patch
>
>
> The {{NativeIO.c#renameTo0}} currently uses {{MoveFile}} Windows system call, which fails
when renaming a file to a destination that already exists.
> This makes the {{TestRollingUpgrade.testRollback}} test fail on Windows, as during that
execution, a DataNode tries to rename block's meta file to a destination that exists.
> The proposal is to change to using {{MoveFileEx}} Windows call, and passing in {{MOVEFILE_REPLACE_EXISTING}}
flag to force the renaming.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message