Return-Path: Delivered-To: apmail-hadoop-core-user-archive@www.apache.org Received: (qmail 86721 invoked from network); 31 Mar 2009 07:30:06 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 31 Mar 2009 07:30:06 -0000 Received: (qmail 59952 invoked by uid 500); 31 Mar 2009 07:30:04 -0000 Delivered-To: apmail-hadoop-core-user-archive@hadoop.apache.org Received: (qmail 59862 invoked by uid 500); 31 Mar 2009 07:30:04 -0000 Mailing-List: contact core-user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: core-user@hadoop.apache.org Delivered-To: mailing list core-user@hadoop.apache.org Received: (qmail 59852 invoked by uid 99); 31 Mar 2009 07:30:04 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 31 Mar 2009 07:30:04 +0000 X-ASF-Spam-Status: No, hits=3.4 required=10.0 tests=HTML_MESSAGE,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [74.125.44.30] (HELO yx-out-2324.google.com) (74.125.44.30) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 31 Mar 2009 07:29:57 +0000 Received: by yx-out-2324.google.com with SMTP id 3so1474008yxj.29 for ; Tue, 31 Mar 2009 00:29:36 -0700 (PDT) MIME-Version: 1.0 In-Reply-To: <4abea26c0903302125nc232210i98e48ea8df0dc899@mail.gmail.com> References: <4abea26c0903302125nc232210i98e48ea8df0dc899@mail.gmail.com> Date: Tue, 31 Mar 2009 00:29:21 -0700 Received: by 10.100.171.10 with SMTP id t10mr4562559ane.53.1238484576263; Tue, 31 Mar 2009 00:29:36 -0700 (PDT) Message-ID: Subject: Re: skip setting output path for a sequential MR job.. From: Aaron Kimball To: core-user@hadoop.apache.org Content-Type: multipart/alternative; boundary=0016368e21683a5e0c0466652922 X-Virus-Checked: Checked by ClamAV on apache.org --0016368e21683a5e0c0466652922 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit You must remove the existing output directory before running the job. This check is put in to prevent you from inadvertently destroying or muddling your existing output data. You can remove the output directory in advance programmatically with code similar to: FileSystem fs = FileSystem.get(conf); // use your JobConf here fs.delete(new Path("/path/to/output/dir"), true); See http://hadoop.apache.org/core/docs/current/api/org/apache/hadoop/fs/FileSystem.htmlfor more details. - Aaron On Mon, Mar 30, 2009 at 9:25 PM, some speed wrote: > Hello everyone, > > Is it necessary to redirect the ouput of reduce to a file? When I am trying > to run the same M-R job more than once, it throws an error that the output > file already exists. I dont want to use command line args so I hard coded > the file name into the program. > > So, Is there a way , I could delete a file on HDFS programatically? > or can i skip setting a output file path n just have my output print to > console? > or can I just append to an existing file? > > > Any help is appreciated. Thanks. > > -Sharath > --0016368e21683a5e0c0466652922--