hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Cutting (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-2066) filenames with ':' colon throws java.lang.IllegalArgumentException
Date Thu, 18 Oct 2007 17:57:50 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-2066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12535994
] 

Doug Cutting commented on HADOOP-2066:
--------------------------------------

Nicholas, yes, perhaps we can optimize our parsing of URIs to accept more cases, so that,
in particular, colons can be included in relative paths, excluding only relative paths that
contain '://'.  Should we still support escapes?  Would you like to make a patch for this?
 Passing tests on Windows will be the hard part.  I've not yet tested my patch on Windows
and assume it will fail.  It would be easier if we don't support escapes...

Lohit, HDFS does not permit colons in file names, while Unix does.  This is hard-coded into
HDFS.  It could perhaps be changed.  It was done to err on the safe side: we did not want
to end up with files whose names were illegal, and hence might, e.g., not be removed or read,
and chose to check for characters which could prove problematic and prohibit them.


> filenames with ':' colon throws java.lang.IllegalArgumentException
> ------------------------------------------------------------------
>
>                 Key: HADOOP-2066
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2066
>             Project: Hadoop
>          Issue Type: Bug
>          Components: dfs
>            Reporter: lohit vijayarenu
>         Attachments: HADOOP-2066.patch
>
>
> File names containing colon ":" throws  java.lang.IllegalArgumentException while LINUX
file system supports it.
> $ hadoop dfs -put ./testfile-2007-09-24-03:00:00.gz filenametest
> Exception in thread "main" java.lang.IllegalArgumentException: java.net.URISyntaxException:
Relative path in absolute
> URI: testfile-2007-09-24-03:00:00.gz
> 	at org.apache.hadoop.fs.Path.initialize(Path.java:140)
> 	at org.apache.hadoop.fs.Path.<init>(Path.java:126)
> 	at org.apache.hadoop.fs.Path.<init>(Path.java:50)
> 	at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:273)
> 	at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:117)
> 	at org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:776)
> 	at org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:757)
> 	at org.apache.hadoop.fs.FsShell.copyFromLocal(FsShell.java:116)
> 	at org.apache.hadoop.fs.FsShell.run(FsShell.java:1229)
> 	at org.apache.hadoop.util.ToolBase.doMain(ToolBase.java:187)
> 	at org.apache.hadoop.fs.FsShell.main(FsShell.java:1342)
> Caused by: java.net.URISyntaxException: Relative path in absolute URI: testfile-2007-09-24-03:00:00.gz
> 	at java.net.URI.checkPath(URI.java:1787)
> 	at java.net.URI.<init>(URI.java:735)
> 	at org.apache.hadoop.fs.Path.initialize(Path.java:137)
> 	... 10 more
> Path(String pathString) when given a filename which contains ':' treats it as URI and
selects anything before ':' as
> scheme, which in this case is clearly not a valid scheme.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message