Return-Path: X-Original-To: apmail-hadoop-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id BFD11D5A5 for ; Wed, 10 Oct 2012 00:19:46 +0000 (UTC) Received: (qmail 38665 invoked by uid 500); 10 Oct 2012 00:19:42 -0000 Delivered-To: apmail-hadoop-user-archive@hadoop.apache.org Received: (qmail 38303 invoked by uid 500); 10 Oct 2012 00:19:42 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 38295 invoked by uid 99); 10 Oct 2012 00:19:42 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 10 Oct 2012 00:19:42 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [209.85.220.176] (HELO mail-vc0-f176.google.com) (209.85.220.176) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 10 Oct 2012 00:19:37 +0000 Received: by mail-vc0-f176.google.com with SMTP id gb22so9210773vcb.35 for ; Tue, 09 Oct 2012 17:19:16 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:x-gm-message-state; bh=sDsjA30OAHGPH/971CUna2bwhhiLZtOEq8IaCRRf/N0=; b=BOB0SPpMoYLp8m6Xr7JAEO9+1mbGwbnmfbDwoFE/hoFs/l4Dm4Rzy7Fm5YE8Em5FBH 9ms+1wX777Ur1hxqphdTcaMdubHDKI/WCXs7uJG8xww1rdpm50U2fZ4PPi3zPzo4xfJd D86b34XjzO9ooE7QKZzt3b2ZtlzntyiwMilmy1TZ5FX7qDkBE28BVacE6TcENrDtCBFu eXrgCF71jR2CwatQMXUZj52vZ1ZOVfL6tg295f5Z7Ylf0eTRk9PjK2WB7mdiY9CgakKs eaVkDq5xMhxzbizgnIrWfOQrEk3brNY5b9Cg36TvCFG8UhtqovE8C9GrlaaE2y5m3HK+ dNWg== MIME-Version: 1.0 Received: by 10.52.71.226 with SMTP id y2mr10346670vdu.74.1349828355759; Tue, 09 Oct 2012 17:19:15 -0700 (PDT) Received: by 10.58.124.227 with HTTP; Tue, 9 Oct 2012 17:19:15 -0700 (PDT) In-Reply-To: References: <2107405151-1349370939-cardhu_decombobulator_blackberry.rim.net-794738392-@b3.c16.bise7.blackberry> Date: Tue, 9 Oct 2012 17:19:15 -0700 Message-ID: Subject: Re: copyFromLocal From: Robert Molina To: user@hadoop.apache.org Content-Type: multipart/alternative; boundary=20cf307f35baa71dea04cba96785 X-Gm-Message-State: ALoCoQmpXQg9VBmlUpUOAW3KwwZ+GtfNDBlbdGEaJvQ7c79VtFJcweWcbUGnQoJdIyhqWwCjl9V6 X-Virus-Checked: Checked by ClamAV on apache.org --20cf307f35baa71dea04cba96785 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Here is the information for webhdfs rest call that should allow you to upload a file: http://hadoop.apache.org/docs/r1.0.3/webhdfs.html#CREATE HTH On Fri, Oct 5, 2012 at 1:16 AM, Visioner Sadak wr= ote: > Hey thanks bejoy and andy act my user just has a desktop web user(like we > browsing web) so i guess first i have to upload the file to my linux > box(wher hadoop is installed) using a web app then frm there i have to us= e > moveFromLocal to put tht file in to hadoop,i thought hadoop can be direct= ly > used to upload to hdfs using web .... > > > On Thu, Oct 4, 2012 at 10:45 PM, Bejoy KS wrote: > >> Hi Sadak >> >> If you are issuing copyFromLocal from a client/edge node you can copy th= e >> files available in the client's lfs to hdfs in cluster. The client/edge >> node could be a box that has all the hadoop jars and config files exactl= y >> same as that of the cluster and the cluster nodes should be accessible f= rom >> this client. >> Regards >> Bejoy KS >> >> Sent from handheld, please excuse typos. >> ------------------------------ >> *From: *"Kartashov, Andy" >> *Date: *Thu, 4 Oct 2012 16:51:35 +0000 >> *To: *user@hadoop.apache.org >> *ReplyTo: *user@hadoop.apache.org >> *Subject: *RE: copyFromLocal >> >> I use =96put =96get commands to bring files in/our of HDFS from/to my h= ome >> directory on EC2. Then use WinSCP to download files to my laptop. >> >> >> >> Andy Kartashov >> >> *MPAC* >> >> Architecture R&D, Co-op >> >> 1340 Pickering Parkway, Pickering, L1V 0C4 >> >> ( Phone : (905) 837 6269 >> >> ( Mobile: (416) 722 1787 >> >> *andy.kartashov@mpac.ca* >> >> >> >> *From:* Visioner Sadak [mailto:visioner.sadak@gmail.com] >> *Sent:* Thursday, October 04, 2012 11:53 AM >> *To:* user@hadoop.apache.org >> *Subject:* copyFromLocal >> >> >> >> guys i have hadoop installled in a remote box ... does copyFromLocal >> method copies data from tht local box only wht if i have to copy data fr= om >> uses desktop pc(for example E drive) thru my my web application will i h= ave >> to first copy data to tht remote box using some java code then use >> copyFromLocal method to copy in to hadoop.... >> NOTICE: This e-mail message and any attachments are confidential, subjec= t >> to copyright and may be privileged. Any unauthorized use, copying or >> disclosure is prohibited. If you are not the intended recipient, please >> delete and contact the sender immediately. Please consider the environme= nt >> before printing this e-mail. AVIS : le pr=E9sent courriel et toute pi=E8= ce >> jointe qui l'accompagne sont confidentiels, prot=E9g=E9s par le droit d'= auteur >> et peuvent =EAtre couverts par le secret professionnel. Toute utilisatio= n, >> copie ou divulgation non autoris=E9e est interdite. Si vous n'=EAtes pas= le >> destinataire pr=E9vu de ce courriel, supprimez-le et contactez imm=E9dia= tement >> l'exp=E9diteur. Veuillez penser =E0 l'environnement avant d'imprimer le = pr=E9sent >> courriel >> > > --20cf307f35baa71dea04cba96785 Content-Type: text/html; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Here is the information for webhdfs rest call that should allow you to uplo= ad a file:=A0
http://hadoop.apache.org/docs/r1.0.3/webhdfs.html#CREATE

HTH

On Fri, Oc= t 5, 2012 at 1:16 AM, Visioner Sadak <visioner.sadak@gmail.com&= gt; wrote:
Hey thanks bejoy and andy act my user just h= as a desktop web user(like we browsing web) so i guess first i have to uplo= ad the file to my linux box(wher hadoop is installed) using a web app then = frm there i have to use moveFromLocal to put tht file in to hadoop,i though= t hadoop can be directly used to upload to hdfs using web ....


On Thu, Oct 4, 2012 at 10:45 PM, Bejoy KS <bejoy.hadoop@gmail.com> wrote:
Hi Sadak

If you a= re issuing copyFromLocal from a client/edge node you can copy the files ava= ilable in the client's lfs to hdfs in cluster. The client/edge node cou= ld be a box that has all the hadoop jars and config files exactly same as t= hat of the cluster and the cluster nodes should be accessible from this cli= ent.
Regards
Bejoy KS

Sent from handheld, please excuse typos.
From: "Kartashov, Andy" <Andy.Kartashov@mpac.ca>
Date: Thu, 4 Oct 2012 16:51:35 +0000
Subject: RE: copyFromLocal

I use =96put =96get commands to= bring files in/our of HDFS from/to my home directory on EC2. Then use WinS= CP to download files to my laptop.

=A0

Andy Kartashov

MPAC

Architecture R&D, Co-op

1340 Pickering Parkway, Pickerin= g, L1V 0C4

( Phone : (905) 837 6269

( Mobile: (416) 722 1787

andy.kartashov@mp= ac.ca

=A0

From: Visioner Sadak [mailto:visioner.sadak@gmail.com]
Sent: Thursday, October 04, 2012 11:53 AM
To: user@hadoop.apache.org<= br>Subject: copyFromLocal

=A0

guys i have hadoop installled in a remote box ... does copyFromLocal met= hod copies data from tht local box only wht if i have to copy data from use= s desktop pc(for example E drive) thru my my web application will i have to= first copy data to tht remote box using some java code then use copyFromLo= cal method to copy in to hadoop....

NOTICE: This e-mail message and any attachments are confidential, sub= ject to copyright and may be privileged. Any unauthorized use, copying or d= isclosure is prohibited. If you are not the intended recipient, please dele= te and contact the sender immediately. Please consider the environment befo= re printing this e-mail. AVIS : le pr=E9sent courriel et toute pi=E8ce join= te qui l'accompagne sont confidentiels, prot=E9g=E9s par le droit d'= ;auteur et peuvent =EAtre couverts par le secret professionnel. Toute utili= sation, copie ou divulgation non autoris=E9e est interdite. Si vous n'= =EAtes pas le destinataire pr=E9vu de ce courriel, supprimez-le et contacte= z imm=E9diatement l'exp=E9diteur. Veuillez penser =E0 l'environneme= nt avant d'imprimer le pr=E9sent courriel


--20cf307f35baa71dea04cba96785--