Return-Path: X-Original-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id E8DA310960 for ; Thu, 7 Nov 2013 06:24:12 +0000 (UTC) Received: (qmail 17322 invoked by uid 500); 7 Nov 2013 06:24:06 -0000 Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org Received: (qmail 17046 invoked by uid 500); 7 Nov 2013 06:24:04 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 17035 invoked by uid 99); 7 Nov 2013 06:24:03 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 07 Nov 2013 06:24:03 +0000 X-ASF-Spam-Status: No, hits=2.7 required=5.0 tests=FREEMAIL_ENVFROM_END_DIGIT,FREEMAIL_REPLY,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of sonalgoyal4@gmail.com designates 209.85.215.43 as permitted sender) Received: from [209.85.215.43] (HELO mail-la0-f43.google.com) (209.85.215.43) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 07 Nov 2013 06:23:59 +0000 Received: by mail-la0-f43.google.com with SMTP id ec20so78610lab.2 for ; Wed, 06 Nov 2013 22:23:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=JtieMRHd5oLURE5yHG+lPc4p539hJFjlp+F8OXJBGOM=; b=KrSl0H6IbNxSeqKiY8ut1XoixryEDkd1/XFCxe2AKtI9xnNvLTGtxfDVy7LHGTLDq0 QEgZt7qvKL9yVP3//RrD438pdYIDXM99UadsiE8QF49b1tOuoue6Piu+7gZBkiNNsPWu SQdgON2GPwWicmsCCBAHLWW2jOYsd1N4NiivK1JmlggDIyjkGH1M9fwBCcUXoAW1jrYQ WOwiydCQQgUekjQNCorFGzmzjFckKhyjeKkMh0tbo0g55LwdhXjAQno/fjdwQQPMyoIA GGQQrCyXSVi674Y8DZM2hUz6WeLzgMmtH4gIs0qhJAqYya/oLGmrFc3IyJIqc8jz1SKY iALQ== MIME-Version: 1.0 X-Received: by 10.112.52.225 with SMTP id w1mr5041574lbo.31.1383805417639; Wed, 06 Nov 2013 22:23:37 -0800 (PST) Received: by 10.114.175.37 with HTTP; Wed, 6 Nov 2013 22:23:37 -0800 (PST) In-Reply-To: References: Date: Thu, 7 Nov 2013 11:53:37 +0530 Message-ID: Subject: Re: Mapper input as argument From: Sonal Goyal To: "user@hadoop.apache.org" Content-Type: multipart/alternative; boundary=001a11c3fe905b57cf04ea904e31 X-Virus-Checked: Checked by ClamAV on apache.org --001a11c3fe905b57cf04ea904e31 Content-Type: text/plain; charset=ISO-8859-1 If you dont need line by line but you want to get a number of lines together, use NLineInputFormat. If you dont want to split at all, override isSplitable in FileInputFormat. Or you can use FileInputFormat, get each line as key/value and compute over it, saving the results and emitting only as necessary. I am not sure what your use case is, but I hope the above helps. Best Regards, Sonal Nube Technologies On Thu, Nov 7, 2013 at 11:44 AM, unmesha sreeveni wrote: > Am i able to get the entire split data from mapper. i dnt need as line by > line. > > my input is of say 50 lines.so these files can be splited into different > mappers right. how to get each split data. are we able to get that data? > > > On Thu, Nov 7, 2013 at 11:39 AM, Sonal Goyal wrote: > >> Hi Unmesha, >> >> What is the computation you are trying to do? If you are interested in >> computing over multiple lines instead of a single line, have a look at >> NLineInputFormat. >> >> Best Regards, >> Sonal >> Nube Technologies >> >> >> >> >> >> >> On Thu, Nov 7, 2013 at 11:35 AM, unmesha sreeveni wrote: >> >>> one more doubt : how to copy each input split entering into mapper into >>> a file for computation? >>> >>> >>> On Thu, Nov 7, 2013 at 10:35 AM, unmesha sreeveni >> > wrote: >>> >>>> My driver code is >>>> FileInputFormat.setInputPaths(job, new Path(args[0])); >>>> FileOutputFormat.setOutputPath(job,new Path(args[1])); >>>> >>>> and My mapper is >>>> public void map(Object key, Text value, Context context) >>>> throws IOException, InterruptedException { >>>> where value.tostring() contains my input data. >>>> >>>> is that a better way to copy all the data s coming into a file and do >>>> computations. OR read each line and do the calculation. >>>> >>>> -- >>>> *Thanks & Regards* >>>> >>>> Unmesha Sreeveni U.B >>>> >>>> *Junior Developer* >>>> >>>> >>>> >>>> >>> >>> >>> -- >>> *Thanks & Regards* >>> >>> Unmesha Sreeveni U.B >>> >>> *Junior Developer* >>> >>> *Amrita Center For Cyber Security * >>> >>> >>> * Amritapuri.www.amrita.edu/cyber/ * >>> >> >> > > > -- > *Thanks & Regards* > > Unmesha Sreeveni U.B > > *Junior Developer* > > *Amrita Center For Cyber Security * > > > * Amritapuri.www.amrita.edu/cyber/ * > --001a11c3fe905b57cf04ea904e31 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
If you dont need line by line but you want to get a number= of lines together, use NLineInputFormat. If you dont want to split at all,= override isSplitable in FileInputFormat. Or you can use FileInputFormat, g= et each line as key/value and compute over it, saving the results and emitt= ing only as necessary.=A0

I am not sure what your use case is, but I hope the above he= lps.

Best Rega= rds,
Sonal
Nube = Technologies=A0






On Thu, Nov 7, 2013 at 11:44 AM, unmesha= sreeveni <unmeshabiju@gmail.com> wrote:
Am i able to get the entire split data from mapper. i dnt need = as line by line.

my input is of say 50 lines.so these files can be splited into differe= nt mappers right. how to get each split data. are we able to get that data?=

On Thu, Nov 7, 2013 at 11:39 AM, Sonal Goy= al <sonalgoyal4@gmail.com> wrote:
Hi Unmesha,

<= div>What is the computation you are trying to do? If you are interested in = computing over multiple lines instead of a single line, have a look at NLin= eInputFormat.=A0

Best Regards,
Sonal
Nube Technologies=A0






On Thu, Nov 7, 2013 at 11:35 AM, unmesha= sreeveni <unmeshabiju@gmail.com> wrote:
one more doubt : how to copy each input split entering into map= per into a file for computation?


On Thu, Nov 7, 2013 at 10:35 AM, unmesha sreeveni <unmeshabiju@gmail.c= om> wrote:
My driver code is=A0
=A0 =A0 =A0 =A0 =A0= =A0 =A0 =A0 FileInputFormat.setInputPaths(job, new Path(args[0]));<= /div>
FileOutputFormat.setOutputPath(job,new P= ath(args[1]));

and My mapper is=A0
=A0 =A0 =A0 =A0 public void map(Object key, Text value, Conte= xt context)
throws IOExcept= ion, InterruptedException {
where value= .tostring() contains my input data.

is that a better way to copy all the data s coming into a file and do comp= utations. OR read each line and do the calculation.

--
Thanks & Regards

Unmesha Sreeveni U.B
Junior Developer





--
=
Thanks & Reg= ards

Unmesha Sreeveni U.B
Junior Developer
Amrita Center For Cyber Security




--
Thanks & Regards

Unmesha Sreeveni U.B
Junior Developer
Amrita Center For Cyber Security

--001a11c3fe905b57cf04ea904e31--