hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From <Shreya....@cognizant.com>
Subject RE: Error calling config method in Mapper
Date Thu, 15 Sep 2011 12:07:26 GMT

Hi Harsh,

 

I resolved the earlier issue I had with mapper but the configure is not
getting called.

 

The signature for Mapper is 

public void map(LongWritable key, Text value, OutputCollector output,
Reporter reporter) throws IOException {

 

I used @override does not seem to work.

Configure is not getting called as I populate some collections there
that are used later in the mapper and I am getting those collections as
blank in the mapper.

 

Regards,

Shreya

 

From: Harsh J [mailto:harsh@cloudera.com] 
Sent: Thursday, September 15, 2011 5:14 PM
To: mapreduce-user@hadoop.apache.org
Subject: Re: Error calling config method in Mapper

 

Hello again Shreya,

 

That signature (for configure) looks right and I think it indeed gets
called. Unsure about your map call issue as I do not see that signature.
Perhaps what you need to resolve this issue by yourself is to make use
of the @Override method annotations in your code. That helps a lot when
dealing with frameworks, such as Hadoop.

 

On Thu, Sep 15, 2011 at 4:40 PM, <Shreya.Pal@cognizant.com> wrote:

Hi

 

I have written a mapper class that has the following structure

 

public class TestMapper extends MapReduceBase implements
Mapper<LongWritable, Text, Text, Text> {

 

 

 

public void configure(JobConf conf) {

Path [] cacheFiles = DistributedCache.getLocalCacheFiles(conf);

            if (null != cacheFiles && cacheFiles.length > 0) {

              for (Path cachePath : cacheFiles) {

                if (cachePath.getName().equals(stopwordCacheName)) {

                  loadStopWords(cachePath);

                  break;

                }

                if (cachePath.getName().equals(positiveCacheName)) {

                        loadPositiveWords(cachePath);

                        break;

                  }

                if (cachePath.getName().equals(negativeCacheName)) {

                        loadNegativeWords(cachePath);

                        break;

                  }

                if (cachePath.getName().equals(searchCacheName)) {

                        loadSearchWords(cachePath);

                        break;

                  }

              }

 

 

//Mapper logic goes here

}

 

Config method should get called before the mapper method by default
right?

But that's not happening..... and the loggers I have used inside mapper
are not getting printed on console and there are no loga in userlogs.
Userlogs is empty. Logger
statements(sLogger.debug("*****************uploaded positive words");)
in the driver class are getting printed on the console though.

 

 

Regards,

Shreya

 

This e-mail and any files transmitted with it are for the sole use of
the intended recipient(s) and may contain confidential and privileged
information. If you are not the intended recipient(s), please reply to
the sender and destroy all copies of the original message. Any
unauthorized review, use, disclosure, dissemination, forwarding,
printing or copying of this email, and/or any action taken in reliance
on the contents of this e-mail is strictly prohibited and may be
unlawful.





 

-- 
Harsh J



This e-mail and any files transmitted with it are for the sole use of the intended recipient(s)
and may contain confidential and privileged information. If you are not the intended recipient(s),
please reply to the sender and destroy all copies of the original message. Any unauthorized
review, use, disclosure, dissemination, forwarding, printing or copying of this email, and/or
any action taken in reliance on the contents of this e-mail is strictly prohibited and may
be unlawful.
Mime
View raw message