hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Devaraj k <devara...@huawei.com>
Subject RE: What happens when I do not output anything from my mapper
Date Mon, 04 Jun 2012 12:38:58 GMT
Hi Murat,

As Praveenesh explained, you can control the map outputs as you want. 

map() function will be called for each input i.e map() function invokes multiple times with
different inputs in the same mapper. You can check by having the logs in the map function
what is happening in it.
   

Thanks
Devaraj

________________________________________
From: praveenesh kumar [praveenesh@gmail.com]
Sent: Monday, June 04, 2012 5:57 PM
To: common-user@hadoop.apache.org
Subject: Re: What happens when I do not output anything from my mapper

You can control your map outputs based on any condition you want. I have
done that - it worked for me.
It could be your code problem that its not working for you.
Can you please share your map code or cross-check whether your conditions
are correct ?

Regards,
Praveenesh

On Mon, Jun 4, 2012 at 5:52 PM, murat migdisoglu <murat.migdisoglu@gmail.com
> wrote:

> Hi,
> I have a small application where I have only mapper class defined(no
> reducer, no combiner).
> Within the mapper class, I have an if condition according to which I decide
> If I want to put something in the context or not.
> If my condition is not match, I want that mapper does not give any output
> to the hdfs.
> But apparently, this does not worj as I expected. Once I run my job, a file
> per mapper in the hdfs with 87 kb of size.
>
> the if block that I'm using in the map method is as following:
> if (ip == null || ip.equals(cip)) {
>            Text value = new Text(mwrapper.toJson());
>            word.set(ip);
>            context.write( word, value);
>        } else {
>            log.info("ip not match [" + ip + "]");
>        }
> }
> }//end of mapper method
>
> How can I manage that? Does mapper always need to have an output?
>
> --
> "Find a job you enjoy, and you'll never work a day in your life."
> Confucius
>

Mime
View raw message