hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Wang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-10684) WebHDFS DataNode calls fail without parameter createparent
Date Thu, 01 Sep 2016 21:36:20 GMT

    [ https://issues.apache.org/jira/browse/HDFS-10684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15456692#comment-15456692
] 

Andrew Wang commented on HDFS-10684:
------------------------------------

Thanks for picking this up John, the patch looks good. Great find here also [~loungerdork],
thanks for the detailed report.

One question, how can we add testing to ensure this type of compatibility for the future?
I assume this gap is because when these parameters were added, WebHDFSFileSystem was also
modified to always send them, and thus was not testing the case where the params were unspecified.

I also thought HttpFSFileSystem as an alternate WebHDFS client implementation would help catch
this. Maybe a test that points HttpFSFileSystem directly at a WebHDFS endpoint?

> WebHDFS DataNode calls fail without parameter createparent
> ----------------------------------------------------------
>
>                 Key: HDFS-10684
>                 URL: https://issues.apache.org/jira/browse/HDFS-10684
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: webhdfs
>    Affects Versions: 2.7.1
>            Reporter: Samuel Low
>            Assignee: John Zhuge
>              Labels: compatibility
>         Attachments: HDFS-10684.001-branch-2.patch
>
>
> Optional boolean parameters that are not provided in the URL cause the WebHDFS create
file command to fail.
>     curl -i -X PUT "http://hadoop-primarynamenode:50070/webhdfs/v1/tmp/test1234?op=CREATE&overwrite=false"
> Response:
>     HTTP/1.1 307 TEMPORARY_REDIRECT
>     Cache-Control: no-cache
>     Expires: Fri, 15 Jul 2016 04:10:13 GMT
>     Date: Fri, 15 Jul 2016 04:10:13 GMT
>     Pragma: no-cache
>     Expires: Fri, 15 Jul 2016 04:10:13 GMT
>     Date: Fri, 15 Jul 2016 04:10:13 GMT
>     Pragma: no-cache
>     Content-Type: application/octet-stream
>     Location: http://hadoop-datanode1:50075/webhdfs/v1/tmp/test1234?op=CREATE&namenoderpcaddress=hadoop-primarynamenode:8020&overwrite=false
>     Content-Length: 0
>     Server: Jetty(6.1.26)
> Following the redirect:
>     curl -i -X PUT -T MYFILE "http://hadoop-datanode1:50075/webhdfs/v1/tmp/test1234?op=CREATE&namenoderpcaddress=hadoop-primarynamenode:8020&overwrite=false"
> Response:
> HTTP/1.1 100 Continue
>     HTTP/1.1 400 Bad Request
>     Content-Type: application/json; charset=utf-8
>     Content-Length: 162
>     Connection: close
>     {"RemoteException":{"exception":"IllegalArgumentException","javaClassName":"java.lang.IllegalArgumentException","message":"Failed
to parse \"null\" to Boolean."}}
> The problem can be circumvented by providing both "createparent" and "overwrite" parameters.
> However, this is not possible when I have no control over the WebHDFS calls, e.g. Ambari
and Hue have errors due to this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message