Return-Path: X-Original-To: apmail-hadoop-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id DD9B410258 for ; Mon, 8 Apr 2013 08:28:41 +0000 (UTC) Received: (qmail 44005 invoked by uid 500); 8 Apr 2013 08:28:36 -0000 Delivered-To: apmail-hadoop-user-archive@hadoop.apache.org Received: (qmail 43628 invoked by uid 500); 8 Apr 2013 08:28:36 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Delivered-To: moderator for user@hadoop.apache.org Received: (qmail 29460 invoked by uid 99); 8 Apr 2013 08:22:57 -0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of marcosm@buscapecompany.com designates 187.95.198.220 as permitted sender) From: MARCOS MEDRADO RUBINELLI To: "user@hadoop.apache.org" Subject: RES: I want to call HDFS REST api to upload a file using httplib. Thread-Topic: I want to call HDFS REST api to upload a file using httplib. Thread-Index: AQHONB7QE2HgYmtgNkG+CPPJpSfn8pjL+fqo Date: Mon, 8 Apr 2013 08:22:25 +0000 Message-ID: References: In-Reply-To: Accept-Language: pt-BR, en-US Content-Language: pt-BR X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [189.54.101.220] Content-Type: multipart/alternative; boundary="_000_B9915517C0BE3447A3754AD79970BE254D6308C8Bpeexcmbx03grup_" MIME-Version: 1.0 X-Virus-Checked: Checked by ClamAV on apache.org --_000_B9915517C0BE3447A3754AD79970BE254D6308C8Bpeexcmbx03grup_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable On your first call, Hadoop will return a URL pointing to a datanode in the = Location header of the 307 response. On your second call, you have to use t= hat URL instead of constructing your own. You can see the specific document= ation here: http://hadoop.apache.org/docs/r1.0.4/webhdfs.html#CREATE Regards, Marcos ________________________________ I want to call HDFS REST api to upload a file using httplib. My program created the file, but no content is in it. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D Here is my code: import httplib conn=3Dhttplib.HTTPConnection("localhost:50070") conn.request("PUT","/webhdfs/v1/levi/4?op=3DCREATE") res=3Dconn.getresponse() print res.status,res.reason conn.close() conn=3Dhttplib.HTTPConnection("localhost:50075") conn.connect() conn.putrequest("PUT","/webhdfs/v1/levi/4?op=3DCREATE&user.name=3Dlevi") conn.endheaders() a_file=3Dopen("/home/levi/4","rb") a_file.seek(0) data=3Da_file.read() conn.send(data) res=3Dconn.getresponse() print res.status,res.reason conn.close() =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Here is the return: 307 TEMPORARY_REDIRECT 201 Created =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D OK, the file was created, but no content was sent. When I comment the #conn.send(data), the result is the same, still no conte= nt. Maybe the file read or the send is wrong, not sure. Do you know how this happened? --_000_B9915517C0BE3447A3754AD79970BE254D6308C8Bpeexcmbx03grup_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
On your first call, Hadoop will return a URL pointing to a data= node in the Location header of the 307 response. On your second call, = you have to use that URL instead of constructing your own. You can see the specific documentation here:

Regards,
Marcos

I want to call HDFS REST api to upload a file using httplib= .

My program created the file, but no content is in it.

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D

Here is my code:

=
import httplib=0A=
=0A=
conn=3Dhttplib=
.HTTPConnection("localhost:50070")=0A=
conn.request("PUT","/webhdfs/v1/levi/4?op=3DCREA=
TE")=0A=
res=3Dconn.getresponse()=0A=
print res.status,res.reason=0A=
conn.close()=0A=
=0A=
conn=3Dhttplib=
.HTTPConnection("localhost:50075")=0A=
conn.connect()=0A=
conn.putrequest("PUT","/webhdfs/v1/levi/4?op=3DC=
REATE&user.name=3Dlevi")=0A=
conn.endheaders()=0A=
a_file=3Dopen<=
span class=3D"pun">("/home/levi/4","rb")=0A=
a_file.seek(0)<=
/span>=0A=
data=3Da_file<=
span class=3D"pun">.read()=0A=
conn.send(data)=
=0A=
res=3Dconn.getresponse()=0A=
print res.status,res.reason=0A=
conn.close()

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D

Here is the return:

307 TEMPORARY_REDIRECT 201 Created

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D

OK, the file was created, but no content was sent.

When I comment the #conn.send(data), the result is the same= , still no content.

Maybe the file read or the send is wrong, not sure.

Do you know how this happened?
--_000_B9915517C0BE3447A3754AD79970BE254D6308C8Bpeexcmbx03grup_--