Return-Path: X-Original-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id CF4EACF9F for ; Mon, 3 Jun 2013 18:59:19 +0000 (UTC) Received: (qmail 47051 invoked by uid 500); 3 Jun 2013 18:59:15 -0000 Delivered-To: apmail-hadoop-hdfs-user-archive@hadoop.apache.org Received: (qmail 46951 invoked by uid 500); 3 Jun 2013 18:59:15 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 46944 invoked by uid 99); 3 Jun 2013 18:59:14 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 03 Jun 2013 18:59:14 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS,WEIRD_PORT X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of dontariq@gmail.com designates 209.85.220.174 as permitted sender) Received: from [209.85.220.174] (HELO mail-vc0-f174.google.com) (209.85.220.174) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 03 Jun 2013 18:59:09 +0000 Received: by mail-vc0-f174.google.com with SMTP id ha12so1025178vcb.33 for ; Mon, 03 Jun 2013 11:58:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; bh=WhMprofjBhEHsyG1WpJamXSZiUfwRB9YV0p/zYOzLIY=; b=WRP32dhREjV+G7XFTcrn16wIfYNMo+6E8+7CsZImdr1oVjgp22BBtNdWWf+nNxeMIy TcDIVz3O0sVBm8DCXEwZjN44MhfvJqin+3BC6lgV64FUa4P0nRa3RFtLQgAeVfdG9wuf FkJgIqP+Q8MXVMhxMtziIEPx+3SlK6x9+EkJWq5USgrGop8WIfSnzTZJ0U7VeFBsTfSa uIWiEnYy3EQVjrtpSkAd08YYbRx1CmRhQgSKB8UNG1+y5Eme63mpBaitR8R6k0/llAFs ILvwsO8CmlNBJxAjv/k3L4BFC1B6+cJf5onWKTFNM/XWRxCue/UfS1V2WhTM8OGEnwqF gICA== X-Received: by 10.52.183.170 with SMTP id en10mr14877443vdc.5.1370285928561; Mon, 03 Jun 2013 11:58:48 -0700 (PDT) MIME-Version: 1.0 Received: by 10.59.12.163 with HTTP; Mon, 3 Jun 2013 11:58:08 -0700 (PDT) In-Reply-To: <3DB84EA5F65C804D98EE9F8562FAA53B513AB51C@ESV4-MBX02.linkedin.biz> References: <9D1A927C-E875-4171-A9B7-F28591D0345C@hortonworks.com> <62AC5F75-8D54-4122-A0D4-BCF1397D4B24@hortonworks.com> <3DB84EA5F65C804D98EE9F8562FAA53B513AB51C@ESV4-MBX02.linkedin.biz> From: Mohammad Tariq Date: Tue, 4 Jun 2013 00:28:08 +0530 Message-ID: Subject: Re: How can we download a file using WebHDFS REST API To: "user@hadoop.apache.org" Content-Type: multipart/alternative; boundary=bcaec548a1b30343cf04de448ecf X-Virus-Checked: Checked by ClamAV on apache.org --bcaec548a1b30343cf04de448ecf Content-Type: text/plain; charset=ISO-8859-1 Hello Mustaqeem, I don't think this is possible through webHDFS, as others have said. But you could use wget(courtesy Datanode API) to download the file. Something like this : wget http://datanode:50075/streamFile/path_of_the_file Please note that you need to issue this command on the DN machine. HTH Warm Regards, Tariq cloudfront.blogspot.com On Fri, May 31, 2013 at 10:39 PM, Adam Faris wrote: > As others have suggested, you need to call "&op=OPEN" and save the > contents into a buffer which you then write to disk. You can do this > with curl by using '-o /my/local/file.txt' in your curl statement. > > -- Adam > > On May 31, 2013, at 10:05 AM, Arpit Gupta > wrote: > > > There is no api to download a file. The client you are using to make > these calls will have to read the contents of the file and save it on your > local file system. > > > > -- > > Arpit Gupta > > Hortonworks Inc. > > http://hortonworks.com/ > > > > On May 31, 2013, at 9:56 AM, Mohammad Mustaqeem <3m.mustaqeem@gmail.com> > wrote: > > > >> This is for reading the content of the file not to download.. > >> I want to download file. > >> > >> > >> On Fri, May 31, 2013 at 10:23 PM, Arpit Gupta > wrote: > >> you will have to use the open operation to read the file and save it. > >> > >> > http://hadoop.apache.org/docs/r2.0.3-alpha/hadoop-project-dist/hadoop-hdfs/WebHDFS.html#Open_and_Read_a_File > >> > >> -- > >> Arpit Gupta > >> Hortonworks Inc. > >> http://hortonworks.com/ > >> > >> On May 30, 2013, at 11:44 PM, Mohammad Mustaqeem < > 3m.mustaqeem@gmail.com> wrote: > >> > >>> Is there any way to download a file from HDFS using WebHDFS REST API? > >>> I have read - > http://hadoop.apache.org/docs/r2.0.3-alpha/hadoop-project-dist/hadoop-hdfs/WebHDFS.html > >>> > >>> But didn't find any way to download file. > >>> > >>> -- > >>> With regards --- > >>> Mohammad Mustaqeem, > >>> M.Tech (CSE) > >>> MNNIT Allahabad > >>> 9026604270 > >>> > >>> > >> > >> > >> > >> > >> -- > >> With regards --- > >> Mohammad Mustaqeem, > >> M.Tech (CSE) > >> MNNIT Allahabad > >> 9026604270 > >> > >> > > > > --bcaec548a1b30343cf04de448ecf Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
Hello Mustaqeem,

=A0 =A0 =A0 =A0 = =A0 I don't think this is possible through webHDFS, as others have said= . But you could use wget(courtesy Datanode API) to download the file. Somet= hing like this :



HTH
<= br clear=3D"all">

On Fri, May 31, 2013 at 10:39 PM, Adam F= aris <afaris@linkedin.com> wrote:
As others have suggested, you need to call "&op=3DOPEN" and s= ave the contents into a buffer which you then write to disk. =A0 =A0You can= do this with curl by using '-o /my/local/file.txt' in your curl st= atement.

-- Adam

On May 31, 2013, at 10:05 AM, Arpit Gupta <arpit@hortonworks.com>
=A0wrote:

> There is no api to download a file. The client you are using to make t= hese calls will have to read the contents of the file and save it on your l= ocal file system.
>
> --
> Arpit Gupta
> Hortonworks Inc.
> http://hortonwor= ks.com/
>
> On May 31, 2013, at 9:56 AM, Mohammad Mustaqeem <3m.mustaqeem@gmail.com> wrote:
>
>> This is for reading the content of the file not to download..
>> I want to download file.
>>
>>
>> On Fri, May 31, 2013 at 10:23 PM, Arpit Gupta <arpit@hortonworks.com> wrote:
>> you will have to use the open operation to read the file and save = it.
>>
>> ht= tp://hadoop.apache.org/docs/r2.0.3-alpha/hadoop-project-dist/hadoop-hdfs/We= bHDFS.html#Open_and_Read_a_File
>>
>> --
>> Arpit Gupta
>> Hortonworks Inc.
>> http://horto= nworks.com/
>>
>> On May 30, 2013, at 11:44 PM, Mohammad Mustaqeem <3m.mustaqeem@gmail.com> wrote:
>>
>>> Is there any way to download a file from HDFS using WebHDFS RE= ST API?
>>> I have read - http:= //hadoop.apache.org/docs/r2.0.3-alpha/hadoop-project-dist/hadoop-hdfs/WebHD= FS.html
>>>
>>> But didn't find any way to download file.
>>>
>>> --
>>> With regards ---
>>> Mohammad Mustaqeem,
>>> M.Tech (CSE)
>>> MNNIT Allahabad
>>> 9026604270
>>>
>>>
>>
>>
>>
>>
>> --
>> With regards ---
>> Mohammad Mustaqeem,
>> M.Tech (CSE)
>> MNNIT Allahabad
>> 9026604270
>>
>>
>


--bcaec548a1b30343cf04de448ecf--