hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-9670) DistCp throws NPE when source is root
Date Thu, 21 Apr 2016 19:46:25 GMT

    [ https://issues.apache.org/jira/browse/HDFS-9670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15252531#comment-15252531
] 

Hudson commented on HDFS-9670:
------------------------------

FAILURE: Integrated in Hadoop-trunk-Commit #9647 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/9647/])
HDFS-9670. DistCp throws NPE when source is root. (John Zhuge via (yzhang: rev a749ba0ceaa843aa83146b6bea19e031c8dc3296)
* hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/SimpleCopyListing.java
* hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestDistCpSystem.java


> DistCp throws NPE when source is root
> -------------------------------------
>
>                 Key: HDFS-9670
>                 URL: https://issues.apache.org/jira/browse/HDFS-9670
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: distcp
>    Affects Versions: 2.6.0
>            Reporter: Yongjun Zhang
>            Assignee: John Zhuge
>              Labels: supportability
>             Fix For: 2.8.0
>
>         Attachments: HDFS-9670.001.patch, HDFS-9670.002.patch
>
>
> Symptom:
> {quote}
> [root@vb0724 ~]# hadoop distcp hdfs://X:8020/ hdfs://Y:8020/
> 16/01/20 11:33:33 INFO tools.DistCp: Input Options: DistCpOptions{atomicCommit=false,
syncFolder=false, deleteMissing=false, ignoreFailures=false, maxMaps=20, sslConfigurationFile='null',
copyStrategy='uniformsize', sourceFileListing=null, sourcePaths=[hdfs://X:8020/], targetPath=hdfs://Y:8020/,
targetPathExists=true, preserveRawXattrs=false, filtersFile='null'}
> 16/01/20 11:33:33 INFO client.RMProxy: Connecting to ResourceManager at Z:8032
> 16/01/20 11:33:33 ERROR tools.DistCp: Exception encountered 
> java.lang.NullPointerException
>   at org.apache.hadoop.tools.util.DistCpUtils.getRelativePath(DistCpUtils.java:144)
>   at org.apache.hadoop.tools.SimpleCopyListing.writeToFileListing(SimpleCopyListing.java:598)
>   at org.apache.hadoop.tools.SimpleCopyListing.writeToFileListingRoot(SimpleCopyListing.java:583)
>   at org.apache.hadoop.tools.SimpleCopyListing.doBuildListing(SimpleCopyListing.java:313)
>   at org.apache.hadoop.tools.SimpleCopyListing.doBuildListing(SimpleCopyListing.java:174)
>   at org.apache.hadoop.tools.CopyListing.buildListing(CopyListing.java:86)
>   at org.apache.hadoop.tools.GlobbedCopyListing.doBuildListing(GlobbedCopyListing.java:90)
>   at org.apache.hadoop.tools.CopyListing.buildListing(CopyListing.java:86)
>   at org.apache.hadoop.tools.DistCp.createInputFileListing(DistCp.java:365)
>   at org.apache.hadoop.tools.DistCp.execute(DistCp.java:171)
>   at org.apache.hadoop.tools.DistCp.run(DistCp.java:122)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>   at org.apache.hadoop.tools.DistCp.main(DistCp.java:429)
> {quote}
> Relevant code:
> {code}
>   private Path computeSourceRootPath(FileStatus sourceStatus,
>                                      DistCpOptions options) throws IOException {
>     Path target = options.getTargetPath();
>     FileSystem targetFS = target.getFileSystem(getConf());
>     final boolean targetPathExists = options.getTargetPathExists();
>     boolean solitaryFile = options.getSourcePaths().size() == 1
>                                                 && !sourceStatus.isDirectory();
>     if (solitaryFile) {
>       if (targetFS.isFile(target) || !targetPathExists) {
>         return sourceStatus.getPath();
>       } else {
>         return sourceStatus.getPath().getParent();
>       }
>     } else {
>       boolean specialHandling = (options.getSourcePaths().size() == 1 && !targetPathExists)
||
>           options.shouldSyncFolder() || options.shouldOverwrite();
>       return specialHandling && sourceStatus.isDirectory() ? sourceStatus.getPath()
:
>           sourceStatus.getPath().getParent();
>     }
>   }
> {code}
> We can see that it could return NULL at the end when doing {{sourceStatus.getPath().getParent()}}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message