Return-Path: X-Original-To: apmail-hadoop-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id D2877DF4E for ; Fri, 14 Sep 2012 11:14:32 +0000 (UTC) Received: (qmail 15987 invoked by uid 500); 14 Sep 2012 11:14:27 -0000 Delivered-To: apmail-hadoop-user-archive@hadoop.apache.org Received: (qmail 15554 invoked by uid 500); 14 Sep 2012 11:14:26 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 15530 invoked by uid 99); 14 Sep 2012 11:14:26 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 14 Sep 2012 11:14:26 +0000 X-ASF-Spam-Status: No, hits=-0.1 required=5.0 tests=FSL_RCVD_USER,HTML_MESSAGE,RCVD_IN_DNSWL_MED,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of hemanty@thoughtworks.com designates 74.125.149.145 as permitted sender) Received: from [74.125.149.145] (HELO na3sys009aog121.obsmtp.com) (74.125.149.145) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 14 Sep 2012 11:14:18 +0000 Received: from mail-vb0-f48.google.com ([209.85.212.48]) (using TLSv1) by na3sys009aob121.postini.com ([74.125.148.12]) with SMTP ID DSNKUFMRdbirpXHnj0irHN7l7E+fiuxtz73A@postini.com; Fri, 14 Sep 2012 04:13:58 PDT Received: by vbme21 with SMTP id e21so3958969vbm.35 for ; Fri, 14 Sep 2012 04:13:57 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:x-gm-message-state; bh=nV6NPSZIRUhp69/CaHbv8egjYyOKtvO+Mrmk68vefks=; b=Jl0MLP7b2oKNiYp41WYbwNxwl+B0yjS8xEyx2ZqQ2L3YxA2b/nHGLOw5NtXb/YfxzH Kb3GKNy4jDdJNPw348yqzMC/pJyBxJpDz7QS4LrB+HyB5fz/suVGeTva+ymd7Xv5m7ZY MM8OCgP+QZhg2sIW4FMqY6tpTcrLgpEL+ht7Yd9BiAcIvsLuUsQxTwiLNipceN/LE7Tq 2j3t5UO3JK5hgl0uN2TMtsPDJK0jrHdr7jXtZ8kn84x0NBK5ooUrkOQbuvJGPuaFr3hz ASFYaIUfqNm0as93e4Ezn/SY94IqRdh9eK+ll1OySe/TOkwJio1A+q8GrsQXSvBWp3gt Se7g== MIME-Version: 1.0 Received: by 10.58.102.48 with SMTP id fl16mr1808867veb.41.1347621236886; Fri, 14 Sep 2012 04:13:56 -0700 (PDT) Received: by 10.58.203.169 with HTTP; Fri, 14 Sep 2012 04:13:56 -0700 (PDT) In-Reply-To: References: Date: Fri, 14 Sep 2012 16:43:56 +0530 Message-ID: Subject: Re: Ignore keys while scheduling reduce jobs From: Hemanth Yamijala To: user@hadoop.apache.org Content-Type: multipart/alternative; boundary=047d7b4190311dcd4404c9a785b1 X-Gm-Message-State: ALoCoQldZbke6gd47LGBjmdtpPnnhcQIUxwNnVem+PyMVb5bbONSKLKhAYXYibug3R/QXFPavSyP X-Virus-Checked: Checked by ClamAV on apache.org --047d7b4190311dcd4404c9a785b1 Content-Type: text/plain; charset=ISO-8859-1 Hi, When do you know the keys to ignore ? You mentioned "after the map stage" .. is this at the end of each map task, or at the end of all map tasks ? Thanks hemanth On Fri, Sep 14, 2012 at 4:36 PM, Aseem Anand wrote: > Hi, > Is there anyway I can ignore all keys except a certain key ( determined > after the map stage) to start only 1 reduce job using a partitioner? If so > could someone suggest such a method. > > Regards, > Aseem > > --047d7b4190311dcd4404c9a785b1 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Hi,

When do you know the keys to ignore ? You mentioned = "after the map stage" .. is this at the end of each map task, or = at the end of all map tasks ?

Thanks
hemanth

On Fri, Sep 14, 2012 at 4:36 PM, = Aseem Anand <aseem.iiith@gmail.com> wrote:
Hi,
Is there anyway I can ignore all keys except a certain key ( determ= ined after the map stage) to start only 1 reduce job using a partitioner? I= f so could someone suggest such a method.

Regards,=
Aseem


--047d7b4190311dcd4404c9a785b1--