hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Lowe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7816) Unable to open webhdfs paths with escape characters
Date Fri, 20 Feb 2015 17:20:13 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14329199#comment-14329199

Jason Lowe commented on HDFS-7816:

This appears to be broken after HDFS-7279.  What's happening is the URI is being properly
encoded by the client but the datanode is not decoding it properly when it tries to act as
a DFS client.

Looks like netty's QueryStringDecoder#path method, despite the javadoc stating it returns
a decoded path, is not returning a decoded path from the URI.  It doesn't use a URI object
to decode it, rather it just performs substrings on the URI string.  Even if you pass it a
URI it uses the raw path, not the decoded path, for the URI and then returns a substring of
that as the path.

As a result the datanode ends up using a non-decoded path and we have a path mismatch between
what the client requested and what the datanode tries to open on their behalf.

> Unable to open webhdfs paths with escape characters
> ---------------------------------------------------
>                 Key: HDFS-7816
>                 URL: https://issues.apache.org/jira/browse/HDFS-7816
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: webhdfs
>    Affects Versions: 2.7.0
>            Reporter: Jason Lowe
>            Priority: Blocker
> webhdfs requests to open files with % characters in the filename fail because the filename
is not being decoded properly.  For example:
> $ hadoop fs -cat 'webhdfs://nn/user/somebody/abc%def'
> cat: File does not exist: /user/somebody/abc%25def

This message was sent by Atlassian JIRA

View raw message