hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Cutting (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-2066) filenames with ':' colon throws java.lang.IllegalArgumentException
Date Mon, 14 Jan 2008 18:04:36 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-2066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12558701#action_12558701
] 

Doug Cutting commented on HADOOP-2066:
--------------------------------------

> I'm not at all sure it makes sense to define what is a valid filename based on a URI
library.

The URI standard provides a good interchange syntax for file names.  But we shouldn't let
it limit what names are possible in various filesystems as we do today: we should support
their full range, using escapes where necessary.  Unfortunately, with the current API, we
can't tell when a character needs to be escaped or when it is intended as a URI meta-character.

The problem is that we construct paths in FileSystem independent code, so we don't know how
to escape things.  Perhaps the solution is to remove the public Path constructor and force
all Paths to be created by a FileSystem#createPath method, so that they can be escaped appropriately.

Thus, when running on Windows, if one passes a string with unescaped backslashes to LocalFileSystem#createPath(),
the backslashes would be interpreted as directory separators, while on Linux or HDFS they'd
be treated as literals.  Unescaped slashes in a Path URI will always be directory separators,
since that's the URI standard we're using for interchange.


> filenames with ':' colon throws java.lang.IllegalArgumentException
> ------------------------------------------------------------------
>
>                 Key: HADOOP-2066
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2066
>             Project: Hadoop
>          Issue Type: Bug
>          Components: dfs
>            Reporter: lohit vijayarenu
>         Attachments: 2066_20071022.patch, HADOOP-2066.patch
>
>
> File names containing colon ":" throws  java.lang.IllegalArgumentException while LINUX
file system supports it.
> $ hadoop dfs -put ./testfile-2007-09-24-03:00:00.gz filenametest
> Exception in thread "main" java.lang.IllegalArgumentException: java.net.URISyntaxException:
Relative path in absolute
> URI: testfile-2007-09-24-03:00:00.gz
> 	at org.apache.hadoop.fs.Path.initialize(Path.java:140)
> 	at org.apache.hadoop.fs.Path.<init>(Path.java:126)
> 	at org.apache.hadoop.fs.Path.<init>(Path.java:50)
> 	at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:273)
> 	at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:117)
> 	at org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:776)
> 	at org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:757)
> 	at org.apache.hadoop.fs.FsShell.copyFromLocal(FsShell.java:116)
> 	at org.apache.hadoop.fs.FsShell.run(FsShell.java:1229)
> 	at org.apache.hadoop.util.ToolBase.doMain(ToolBase.java:187)
> 	at org.apache.hadoop.fs.FsShell.main(FsShell.java:1342)
> Caused by: java.net.URISyntaxException: Relative path in absolute URI: testfile-2007-09-24-03:00:00.gz
> 	at java.net.URI.checkPath(URI.java:1787)
> 	at java.net.URI.<init>(URI.java:735)
> 	at org.apache.hadoop.fs.Path.initialize(Path.java:137)
> 	... 10 more
> Path(String pathString) when given a filename which contains ':' treats it as URI and
selects anything before ':' as
> scheme, which in this case is clearly not a valid scheme.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message