hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dmytro Molkov (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-1109) HFTP and URL Encoding
Date Thu, 18 Nov 2010 21:31:15 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12933573#action_12933573
] 

Dmytro Molkov commented on HDFS-1109:
-------------------------------------

Suresh, the reason for this patch was the way url encoding,decoding works.
We had files that had a whitespace in the name, when the request got to the datanode the filename
property had a '+' instead of the whitespace (it was url encoded but not decoded). As a result
the datanode couldn't read that file and the HFTP call failed.
Everywhere else we are using the URL path to pass the hdfs path around except for this case.
Changing this one place deals with the url encoding and the path of the file is passed around
properly in the HFTP protocol. Hope that answers your question.

> HFTP and URL Encoding
> ---------------------
>
>                 Key: HDFS-1109
>                 URL: https://issues.apache.org/jira/browse/HDFS-1109
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: contrib/hdfsproxy, data-node
>    Affects Versions: 0.20.1, 0.20.2, 0.20.3, 0.21.0, 0.22.0
>            Reporter: Dmytro Molkov
>            Assignee: Dmytro Molkov
>             Fix For: 0.22.0
>
>         Attachments: HDFS-1109.2.patch, HDFS-1109.2_y0.20.1xx.patch, HDFS-1109.2_y0.20.1xx_incremental.patch,
HDFS-1109.patch
>
>
> We just saw this error happen in our cluster. If there is a file that has a "+" sign
in the name it is not readable through HFTP protocol.
> The problem is when we are reading a file with HFTP we are passing a name of the file
as a parameter in request and + gets undecoded into space on the server side. So the datanode
receiving the streamFile request tries to access a file with space instead of + in the name
and doesn't find that file.
> The proposed solution is to pass the filename as a part of URL as with all the other
HFTP commands, since this is the only place where it is not being treated this way. Are there
any objections to this?

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message