hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "小学园PHP" <xxy-...@qq.com>
Subject Re:RES: I want to call HDFS REST api to upload a file using httplib.
Date Tue, 09 Apr 2013 04:05:17 GMT
Really Thanks.
But the returned URL is wrong. And the localhost is the real URL, as i tested successfully
with curl using "localhost".
Can anybody help me translate the curl to Python httplib?
curl -i -X PUT -T <LOCAL_FILE> "http://<DATANODE>:<PORT>/webhdfs/v1/<PATH>?op=CREATE"
I test it using python httplib, and receive the right response. But the file uploaded to HDFS
is empty, no data sent!!
Is "conn.send(data)"  the problem?

------------------ Original ------------------
From:  "MARCOS MEDRADO RUBINELLI"<marcosm@buscapecompany.com>;
Date:  Mon, Apr 8, 2013 04:22 PM
To:  "user@hadoop.apache.org"<user@hadoop.apache.org>; 

Subject:  RES: I want to call HDFS REST api to upload a file using httplib.

 On your first call, Hadoop will return a URL pointing to a datanode in the Location header
of the 307 response. On your second call, you have to use that URL instead of constructing
 your own. You can see the specific documentation here: http://hadoop.apache.org/docs/r1.0.4/webhdfs.html#CREATE
I want to call HDFS REST api to upload a file using httplib.
My program created the file, but no content is in it.
Here is my code:
 import httplib conn=httplib.HTTPConnection("localhost:50070") conn.request("PUT","/webhdfs/v1/levi/4?op=CREATE")
res=conn.getresponse() print res.status,res.reason conn.close() conn=httplib.HTTPConnection("localhost:50075")
conn.connect() conn.putrequest("PUT","/webhdfs/v1/levi/4?op=CREATE&user.name=levi") conn.endheaders()
a_file=open("/home/levi/4","rb") a_file.seek(0) data=a_file.read() conn.send(data) res=conn.getresponse()
print res.status,res.reason conn.close() 
Here is the return:
OK, the file was created, but no content was sent.
When I comment the #conn.send(data), the result is the same, still no content.
Maybe the file read or the send is wrong, not sure.
Do you know how this happened?
View raw message