hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Nauroth (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-8761) Windows HDFS daemon - datanode.DirectoryScanner: Error compiling report (...) XXX is not a prefix of YYY
Date Mon, 03 Aug 2015 21:44:04 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14652581#comment-14652581

Chris Nauroth commented on HDFS-8761:

[~odelalleau], glad to hear this helped!

bq. I wonder how this is not a bug though, even if there exists a workaround. But not a big

I agree that the configuration file can end up looking non-intuitive on Windows.  Unfortunately,
I don't see a way to do any better while maintaining the feature that everything defaults
to using {{hadoop.tmp.dir}} for quick dev deployments.  This is a side effect of the fact
that a Windows file system path is not always valid as a URL.  On Linux, a file system path
will always be a valid URL (assuming the individual path names stick to the characters that
don't require escaping).  I typically advise using a full {{file:}} URL in production configurations
to make everything clearer for operators.

> Windows HDFS daemon - datanode.DirectoryScanner: Error compiling report (...) XXX is
not a prefix of YYY
> --------------------------------------------------------------------------------------------------------
>                 Key: HDFS-8761
>                 URL: https://issues.apache.org/jira/browse/HDFS-8761
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: HDFS
>    Affects Versions: 2.7.1
>         Environment: Windows 7, Java SDK 1.8.0_45
>            Reporter: Olivier Delalleau
>            Priority: Minor
> I'm periodically seeing errors like the one below output by the HDFS daemon (started
with start-dfs.cmd). This is with the default settings for data location (=not specified in
my hdfs-site.xml). I assume it may be fixable by specifying a path with the drive letter in
the config file, however I haven't be able to do that (see http://stackoverflow.com/questions/31353226/setting-hadoop-tmp-dir-on-windows-gives-error-uri-has-an-authority-component).
> 15/07/11 17:29:57 ERROR datanode.DirectoryScanner: Error compiling report
> java.util.concurrent.ExecutionException: java.lang.RuntimeException: \tmp\hadoop-odelalleau\dfs\data
is not a prefix of D:\tmp\hadoop-odelalleau\dfs\data\current\BP-1474392971-\current\finalized\subdir0\subdir0\blk_1073741825
>         at java.util.concurrent.FutureTask.report(FutureTask.java:122)
>         at java.util.concurrent.FutureTask.get(FutureTask.java:192)
>         at org.apache.hadoop.hdfs.server.datanode.DirectoryScanner.getDiskReport(DirectoryScanner.java:566)
>         at org.apache.hadoop.hdfs.server.datanode.DirectoryScanner.scan(DirectoryScanner.java:425)
>         at org.apache.hadoop.hdfs.server.datanode.DirectoryScanner.reconcile(DirectoryScanner.java:406)
>         at org.apache.hadoop.hdfs.server.datanode.DirectoryScanner.run(DirectoryScanner.java:362)
>         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)

This message was sent by Atlassian JIRA

View raw message