Return-Path: X-Original-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 1E7521002F for ; Fri, 25 Oct 2013 12:52:36 +0000 (UTC) Received: (qmail 41881 invoked by uid 500); 25 Oct 2013 12:52:24 -0000 Delivered-To: apmail-hadoop-hdfs-user-archive@hadoop.apache.org Received: (qmail 41745 invoked by uid 500); 25 Oct 2013 12:52:23 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 41737 invoked by uid 99); 25 Oct 2013 12:52:21 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 25 Oct 2013 12:52:21 +0000 X-ASF-Spam-Status: No, hits=2.5 required=5.0 tests=FREEMAIL_REPLY,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of shahab.yunus@gmail.com designates 74.125.83.44 as permitted sender) Received: from [74.125.83.44] (HELO mail-ee0-f44.google.com) (74.125.83.44) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 25 Oct 2013 12:52:17 +0000 Received: by mail-ee0-f44.google.com with SMTP id c4so1437768eek.17 for ; Fri, 25 Oct 2013 05:51:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=GCGO+2hgdDzV+YxZMfZ+R5x9tStWVLO83UXwKqG154w=; b=Ds44W4OSs4/dZ1TD+QwFi88LJOhT5XVphROlumbwlDwDSrZdqMWE9lXUTH1pJxL51n eE1G2Nr72i24OKT411A2/4kmA9QD9ZAuarJPTuivfwqt03QxE465t2pTJwquAkN9sMww /vpSUy7Z2foCl8VqC5I59IZsusvMAtKErTt3fjvnYsErlscvZEef3aPkdO38Bucsobux xFzqNAW9+seQcbiNcBRLgz7bIcR3EDsNGycBiVFZc5yPw/BrwpWGdG1GmcM0INYGHIlp Cig6wKg+iHoZch0CWG/8rtM79BuYgeYu3l1h/k23bQpul6xqyy6UWZU6PSEp7EWFlGlg dPlw== MIME-Version: 1.0 X-Received: by 10.205.10.200 with SMTP id pb8mr3441654bkb.16.1382705516472; Fri, 25 Oct 2013 05:51:56 -0700 (PDT) Received: by 10.205.8.197 with HTTP; Fri, 25 Oct 2013 05:51:56 -0700 (PDT) In-Reply-To: <1382671535.36598.YahooMailNeo@web120906.mail.ne1.yahoo.com> References: <1382653707.36331.YahooMailNeo@web120901.mail.ne1.yahoo.com> <1382671535.36598.YahooMailNeo@web120906.mail.ne1.yahoo.com> Date: Fri, 25 Oct 2013 08:51:56 -0400 Message-ID: Subject: Re: Mapreduce outputs to a different cluster? From: Shahab Yunus To: "user@hadoop.apache.org" , "S. Zhou" Content-Type: multipart/alternative; boundary=20cf301cc62a239df604e9903713 X-Virus-Checked: Checked by ClamAV on apache.org --20cf301cc62a239df604e9903713 Content-Type: text/plain; charset=ISO-8859-1 You can specify the HDFS path as follows: FileOutputFormat.setOutputPath(conf, new Path(args[1])); where Path object is of course the location of your output dir. See this for details http://www.rohitmenon.com/index.php/introducing-mapreduce-part-i/ Regards, Shahab On Thu, Oct 24, 2013 at 11:25 PM, S. Zhou wrote: > Thanks Shahab & Yong. If cluster B (in which I want to dump output) has > url "hdfs://machine.domain:8080" and data folder "/tmp/myfolder", what > should I specify as the output path for MR job? > Thanks > > > On Thursday, October 24, 2013 5:31 PM, java8964 java8964 < > java8964@hotmail.com> wrote: > Just specify the output location using the URI to another cluster. As > long as the network is accessible, you should be fine. > > Yong > > ------------------------------ > Date: Thu, 24 Oct 2013 15:28:27 -0700 > From: myxjtu@yahoo.com > Subject: Mapreduce outputs to a different cluster? > To: user@hadoop.apache.org > > The scenario is: I run mapreduce job on cluster A (all source data is in > cluster A) but I want the output of the job to cluster B. Is it possible? > If yes, please let me know how to do it. > > Here are some notes of my mapreduce job: > 1. the data source is an HBase table > 2. It only has mapper no reducer. > > Thanks > Senqiang > > > > --20cf301cc62a239df604e9903713 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
You can specify the HDFS path as follows:
FileOutputFormat.se= tOutputPath(conf, new Path(args[1]));
where Path object is of course the location of your output dir.
See this for details

Regards,
Shahab


On Thu,= Oct 24, 2013 at 11:25 PM, S. Zhou <myxjtu@yahoo.com> wrote:<= br>
Thanks Shahab & Yong. If clus= ter B (in which I want to dump output) has url "hdfs://machine.domain:= 8080" and data folder "/tmp/myfolder", what should I specify= as the output path for MR job?
Thanks


<= div style=3D"font-family:HelveticaNeue,Helvetica Neue,Helvetica,Arial,Lucid= a Grande,sans-serif;font-size:12pt">
On Thursday, October 24, 2013 5:31= PM, java8964 java8964 <java8964@hotmail.com> wrote:
Just specify the output location using the URI to another = cluster. As long as the network is accessible, you should be fine.

Yong

Date: Thu, 24 Oct 2013 15:28:27 -0700
From: myxjtu@yahoo.com
Subject: Mapreduce outputs to a different cluster?
T= o: user@hadoop.= apache.org

The scenario i= s: I run mapreduce job on cluster A (all source data is in cluster A) but I= want the output of the job to cluster B. Is it possible? If yes, please le= t me know how to do it.

Here are some notes of my mapreduce = job:
1. the data source is an HBase table
2. It only has mapper no reducer.

Thanks
Senqiang



--20cf301cc62a239df604e9903713--