hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adam Faris <afa...@linkedin.com>
Subject Re: Put a file to HDFS with WebHDFS ?
Date Wed, 20 Mar 2013 20:42:49 GMT
The WebHDFS wiki has bad information regarding curl.  According to RFC 2616 Section 10.3.8,
 a web client shall not automatically follow a 307 redirect unless the response is from a
GET or HEAD request.   As jetty returns a 307 response for the PUT request against the WebHDFS
api, curl does the proper thing and does not automatically follow the redirect.  You need
to add "-L" to override the default behavior in curl, forcing it to follow the Location header
in the HTTP response.   Here's an example from a secure grid.  

$ curl --negotiate -u : -i -X PUT -T /etc/motd -L 'http://mynamenode.example.com:50070/webhdfs/v1/tmp/motd?op=CREATE'
HTTP/1.1 100 Continue

HTTP/1.1 401 Unauthorized
Content-Type: text/html; charset=utf-8
WWW-Authenticate: Negotiate
Content-Length: 0
Server: Jetty(6.1.26)

HTTP/1.1 100 Continue

HTTP/1.1 307 TEMPORARY_REDIRECT
Content-Type: application/octet-stream
Expires: Thu, 01-Jan-1970 00:00:00 GMT
Set-Cookie: hadoop.auth="xxxxx";Path=/
Location: http://datanode1.example.com:70/webhdfs/v1/tmp/motd?op=CREATE&delegation=xxxxx&overwrite=false
Content-Length: 0
Server: Jetty(6.1.26)

HTTP/1.1 100 Continue

HTTP/1.1 201 Created
Content-Type: application/octet-stream
Location: webhdfs://mynamenode.example.com:50070/tmp/motd
Content-Length: 0
Server: Jetty(6.1.26)

-- Good luck, Adam

On Mar 19, 2013, at 11:19 PM, ken barclay <ken.barclay@att.net> wrote:

> It didn't fail. If you hit the URL it returned, it will do the create.
> 
> --- On Wed, 3/20/13, 小学园PHP <xxy-php@qq.com> wrote:
> 
> From: 小学园PHP <xxy-php@qq.com>
> Subject: Put a file to HDFS with WebHDFS ?
> To: "user" <user@hadoop.apache.org>
> Date: Wednesday, March 20, 2013, 3:24 AM
> 
> When i use the WebHDFS, i wanna put a file to Hadoop.
> my curl command:
> curl -i -X PUT -T /home/levi/3 "http://localhost:50070/webhdfs/v1/levi/3?op=CREATE"
> 
> But i got a return:
> HTTP/1.1 100 Continue
> 
> HTTP/1.1 307 TEMPORARY_REDIRECT
> Content-Type: application/octet-stream
> Location: http://58.53.211.46:50075/webhdfs/v1/levi/3?op=CREATE&overwrite=false
> Content-Length: 0
> Server: Jetty(6.1.26)
> 
> OK, it failed.
> Do somebody know what has happened?

Mime
View raw message