Return-Path: X-Original-To: apmail-hadoop-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id BA44FDE84 for ; Sun, 30 Dec 2012 19:39:26 +0000 (UTC) Received: (qmail 75654 invoked by uid 500); 30 Dec 2012 19:39:22 -0000 Delivered-To: apmail-hadoop-user-archive@hadoop.apache.org Received: (qmail 75525 invoked by uid 500); 30 Dec 2012 19:39:21 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 75517 invoked by uid 99); 30 Dec 2012 19:39:21 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 30 Dec 2012 19:39:21 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (nike.apache.org: local policy) Received: from [209.85.219.43] (HELO mail-oa0-f43.google.com) (209.85.219.43) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 30 Dec 2012 19:39:13 +0000 Received: by mail-oa0-f43.google.com with SMTP id k1so11185483oag.2 for ; Sun, 30 Dec 2012 11:38:52 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:sender:x-originating-ip:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:content-type :x-gm-message-state; bh=o9iM6HbwvkXsVQYILbL5ceVs7hNgUodS6617fMbRi3A=; b=oIi4RUFUDTKOUpOJ3YHRYTOfLK6TT03iGRO4rp22bz7QNfzIQsCU3el+REsldH3Rk8 x+CWDawMcg2bc9dvFZa0v3Sb1I4ynMon/kYdgeKDi7d8FwEVa3WXW7eVcuXsBto/L1Zp sgW2zHDFd6LrRehtvEQNGBo33L6JalMk2Jex4GU+FW7koTpvBvYvEgKlTZMIrZTQHGwQ MglwF0M6zkRd0YouBhz2GAqS2HIZ2IZDZ08H4s65yCSPxQD/Uj1CFHtu8xyC1BUDbC1m YKu1f4qS97oQWxsCLHGcxPC3fwlcZkpNg2L3zwcUIRvonoVPDHwu5hJWDeJWjBmyaOlI drjQ== MIME-Version: 1.0 Received: by 10.182.188.36 with SMTP id fx4mr31930859obc.6.1356896332591; Sun, 30 Dec 2012 11:38:52 -0800 (PST) Sender: niels@basj.es Received: by 10.76.27.225 with HTTP; Sun, 30 Dec 2012 11:38:52 -0800 (PST) X-Originating-IP: [80.100.47.45] Received: by 10.76.27.225 with HTTP; Sun, 30 Dec 2012 11:38:52 -0800 (PST) In-Reply-To: References: Date: Sun, 30 Dec 2012 20:38:52 +0100 X-Google-Sender-Auth: L547y9dap7FB_pf3Nsc9GH6xvwI Message-ID: Subject: Re: What is the preferred way to pass a small number of configuration parameters to a mapper or reducer From: Niels Basjes To: user@hadoop.apache.org Content-Type: multipart/alternative; boundary=f46d04463168e6b0dc04d2170b9c X-Gm-Message-State: ALoCoQl7B8dhWqNhTk/dzoaycyxObyBWCOmdasskKN77BX1tUV0+QKJNegKFaaqjyE4DDqsm7Cp3 X-Virus-Checked: Checked by ClamAV on apache.org --f46d04463168e6b0dc04d2170b9c Content-Type: text/plain; charset=ISO-8859-1 F. put a mongodb replica set on all hadoop workernodes and let the tasks query the mongodb at localhost. (this is what I did recently with a multi GiB dataset) -- Met vriendelijke groet, Niels Basjes (Verstuurd vanaf mobiel ) Op 30 dec. 2012 20:01 schreef "Jonathan Bishop" het volgende: > E. Store them in hbase... > > > On Sun, Dec 30, 2012 at 12:24 AM, Hemanth Yamijala < > yhemanth@thoughtworks.com> wrote: > >> If it is a small number, A seems the best way to me. >> >> On Friday, December 28, 2012, Kshiva Kps wrote: >> >>> >>> Which one is current .. >>> >>> >>> What is the preferred way to pass a small number of configuration >>> parameters to a mapper or reducer? >>> >>> >>> >>> >>> >>> *A. *As key-value pairs in the jobconf object. >>> >>> * * >>> >>> *B. *As a custom input key-value pair passed to each mapper or >>> reducer. >>> >>> * * >>> >>> *C. *Using a plain text file via the Distributedcache, which each >>> mapper or reducer reads. >>> >>> * * >>> >>> *D. *Through a static variable in the MapReduce driver class (i.e., >>> the class that submits the MapReduce job). >>> >>> >>> >>> *Answer: B* >>> >>> >>> >> > --f46d04463168e6b0dc04d2170b9c Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable

F. put a mongodb replica set on all hadoop workernodes and let the tasks= query the mongodb at localhost.

(this is what I did recently with a multi GiB dataset)

--
Met vriendelijke groet,
Niels Basjes
(Verstuurd vanaf mobiel )

Op 30 dec. 2012 20:01 schreef "Jonathan Bis= hop" <jbishop.rwc@gmail.co= m> het volgende:
E. Store them in hbase...
=

On Sun, Dec 30, 2012 at 12:24 AM, Hemant= h Yamijala <yhemanth@thoughtworks.com> wrote:
If it is a small number, A seems the best wa= y to me.

On Friday, December 28, 2012, Kshiva Kps wrot= e:


Which one is current ..


What is the preferred way to pass a small number of configuration parameters to a mapper or reducer?

=A0

=A0

<= b>A.=A0 = As key-value pairs = in the jobconf object.

=A0

<= b>B.=A0 = As a custom input k= ey-value pair passed to each mapper or reducer.

=A0

<= b>C.=A0 = Using a plain text = file via the Distributedcache, which each mapper or reducer reads.

=A0

D.=A0 <= /span>Through a static variable in the MapReduce driver class (i.e., the class that submits the MapReduce job).

=A0

Answer: B

=A0


--f46d04463168e6b0dc04d2170b9c--