Return-Path: X-Original-To: apmail-hadoop-common-user-archive@www.apache.org Delivered-To: apmail-hadoop-common-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 1B67CE430 for ; Thu, 10 Jan 2013 12:45:14 +0000 (UTC) Received: (qmail 15307 invoked by uid 500); 10 Jan 2013 12:45:08 -0000 Delivered-To: apmail-hadoop-common-user-archive@hadoop.apache.org Received: (qmail 15205 invoked by uid 500); 10 Jan 2013 12:45:08 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 15198 invoked by uid 99); 10 Jan 2013 12:45:08 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 10 Jan 2013 12:45:08 +0000 X-ASF-Spam-Status: No, hits=-0.1 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_MED,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of hemanty@thoughtworks.com designates 64.18.0.143 as permitted sender) Received: from [64.18.0.143] (HELO exprod5og102.obsmtp.com) (64.18.0.143) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 10 Jan 2013 12:45:01 +0000 Received: from mail-ob0-f199.google.com ([209.85.214.199]) (using TLSv1) by exprod5ob102.postini.com ([64.18.4.12]) with SMTP ID DSNKUO63t5Kx6VM1eNpAMUalpGKonCeRaMyX@postini.com; Thu, 10 Jan 2013 04:44:40 PST Received: by mail-ob0-f199.google.com with SMTP id 16so2426062obc.10 for ; Thu, 10 Jan 2013 04:44:39 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:mime-version:in-reply-to:references:date:message-id :subject:from:to:content-type:x-gm-message-state; bh=w9cpFZlIDhjEIobAvpH+DekkI4GbLVAkRX/Aar1Yj+8=; b=Uq6Jm/3qXrGnruWOtEDou4tytPrZJ1qFpmCbartrsul0vE1t/BjCNVh+/96AFc+h8P lw3D4tFJfXd880kd1BXRJZb3JmfMnxgtvEDOI4y1yyymvoWpp/nDR8OiNae3AwrQgh86 +WnIvGUVQXiUV+4ehgNWziTeXyVb7yJ19TLTVdXcUrlVyQFi7mqQOWX4rZJUEUhFsOuu J2zVcV63Qcl1XgFqQ9QU3hcrNt0VLYEQQnqiHvnywenZTp6DE3L1sLDHAxF2TIInIHmV kFwZWNmCgKLdD0ABv0V39N/f7RYK6x9h+7+XYiLvevixHf3fM095P/8Jveks3gKS6ohC hh1Q== X-Received: by 10.182.52.105 with SMTP id s9mr51454822obo.25.1357821879063; Thu, 10 Jan 2013 04:44:39 -0800 (PST) MIME-Version: 1.0 Received: by 10.182.52.105 with SMTP id s9mr51454816obo.25.1357821878958; Thu, 10 Jan 2013 04:44:38 -0800 (PST) Received: by 10.76.1.18 with HTTP; Thu, 10 Jan 2013 04:44:38 -0800 (PST) In-Reply-To: References: Date: Thu, 10 Jan 2013 18:14:38 +0530 Message-ID: Subject: Re: Not committing output in map reduce From: Hemanth Yamijala To: "user@hadoop.apache.org" Content-Type: multipart/alternative; boundary=14dae9399429c3793a04d2ee8aa1 X-Gm-Message-State: ALoCoQkgXpkHTcSOS3doHULJhI7Wxe8sI0a4zOMX1jP0zeVmK6pGsz7ivFus0Qk4vo4Vub6P0MscClqkUteDuwGW1liGTSyS0CK/tSqEKkYc3TItLlYS/weZ+VNP1TH5COz02H8AMUOAsv+o2B9pYFvRZ07z6/CsPQ== X-Virus-Checked: Checked by ClamAV on apache.org --14dae9399429c3793a04d2ee8aa1 Content-Type: text/plain; charset=ISO-8859-1 Is this the same as: http://stackoverflow.com/questions/6137139/how-to-save-only-non-empty-reducers-output-in-hdfs? i.e. LazyOutputFormat, etc. ? On Thu, Jan 10, 2013 at 4:51 PM, Pratyush Chandra < chandra.pratyush@gmail.com> wrote: > Hi, > > I am using s3n as file system. I do not wish to create output folders and > file if there is no output from RecordReader implementation. Currently it > creates empty part* files and _SUCCESS. Is there a way to do so ? > > -- > Pratyush Chandra > --14dae9399429c3793a04d2ee8aa1 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Is this the same as:=A0http://stackoverflow.= com/questions/6137139/how-to-save-only-non-empty-reducers-output-in-hdfs ?

i.e. LazyOutputFormat, etc. ?


On Thu, Jan 10, 2013 at 4:51 PM, Praty= ush Chandra <chandra.pratyush@gmail.com> wrote:
Hi,

I am using s3n as file system. I = do not wish to create output folders and file if there is no output from Re= cordReader implementation. Currently it creates empty part* files and _SUCC= ESS. Is there a way to do so ?

--
Pratyush Chandra

--14dae9399429c3793a04d2ee8aa1--