Return-Path: Delivered-To: apmail-hadoop-core-dev-archive@www.apache.org Received: (qmail 62906 invoked from network); 21 Apr 2008 05:12:20 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 21 Apr 2008 05:12:20 -0000 Received: (qmail 92042 invoked by uid 500); 21 Apr 2008 05:12:20 -0000 Delivered-To: apmail-hadoop-core-dev-archive@hadoop.apache.org Received: (qmail 92024 invoked by uid 500); 21 Apr 2008 05:12:20 -0000 Mailing-List: contact core-dev-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: core-dev@hadoop.apache.org Delivered-To: mailing list core-dev@hadoop.apache.org Received: (qmail 92009 invoked by uid 99); 21 Apr 2008 05:12:20 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 20 Apr 2008 22:12:20 -0700 X-ASF-Spam-Status: No, hits=1.2 required=10.0 tests=SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (nike.apache.org: local policy) Received: from [203.99.254.143] (HELO rsmtp1.corp.hki.yahoo.com) (203.99.254.143) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 21 Apr 2008 05:11:24 +0000 Received: from [10.66.74.35] (sevenshade-lr.eglbp.corp.yahoo.com [10.66.74.35]) by rsmtp1.corp.hki.yahoo.com (8.13.8/8.13.8/y.rout) with ESMTP id m3L5BWLN077345 for ; Sun, 20 Apr 2008 22:11:32 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; s=serpent; d=yahoo-inc.com; c=nofws; q=dns; h=message-id:date:from:user-agent:mime-version:to:subject: references:in-reply-to:content-type:content-transfer-encoding; b=lkmcsoGLTh09nsNq6lG/lmKV8n9SOS6+JPxEz7mBnwEi9/lXMLtNONI9DUaAll1N Message-ID: <480C2203.3070807@yahoo-inc.com> Date: Mon, 21 Apr 2008 10:41:31 +0530 From: Amar Kamat User-Agent: Thunderbird 2.0.0.12 (X11/20080213) MIME-Version: 1.0 To: core-dev@hadoop.apache.org Subject: Re: Output filename generation? References: <1b29507a0804180729y61948638ue651b1d10f674559@mail.gmail.com> In-Reply-To: <1b29507a0804180729y61948638ue651b1d10f674559@mail.gmail.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org pi song wrote: > Dear hadoop mailling-list, > > Is there a way to control output filename generation? A sample use case is > when I want 2 MapReduce jobs to output to the same directory. > I think you need to write your own output format (see http://tinyurl.com/4aszgk). Look at OutputFormat.getRecordWriter(). The parameter *name* is what determines the output filename. One easy way would be to append the job-name to this *name* in OutputFormat.getRecordWriter(). Something like public RecordWriter getRecordWriter(FileSystem ignored, JobConf job, String name, Progressable progress) throws IOException { name = name + "_" + job.getJobName(); //rest of the code .. taken from Hadoop-0.16.3 } Amar > Pi > >