hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gino Gu01 <Gino_G...@infosys.com>
Subject RE: An issue in MapReduce Tutorial
Date Mon, 24 Nov 2014 09:42:07 GMT
Adding  the link to the tutorial.
http://hadoop.apache.org/docs/current/hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduceTutorial.html

From: Gino Gu01
Sent: Monday, November 24, 2014 5:29 PM
To: 'user@hadoop.apache.org'
Subject: An issue in MapReduce Tutorial

There is one bug in WordCount v2.0 which is part of MapReduce Tutorial.
How to reproduce:

Run the application:

$ bin/hadoop jar wc.jar WordCount2 /user/joe/wordcount/input /user/joe/wordcount/output
It will throw Null Pointer Exception during map phase.

Reason:
Below highlighted line set the default value of wordcount.skip.patterns to true.
But in the arguments we didn't pass the any patterns file, so the line for (URI patternsURI
: patternsURIs) throws exception.
    public void setup(Context context) throws IOException,
        InterruptedException {
      conf = context.getConfiguration();
      caseSensitive = conf.getBoolean("wordcount.case.sensitive", true);
      if (conf.getBoolean("wordcount.skip.patterns", true)) {
        URI[] patternsURIs = Job.getInstance(conf).getCacheFiles();
        for (URI patternsURI : patternsURIs) {
          Path patternsPath = new Path(patternsURI.getPath());
          String patternsFileName = patternsPath.getName().toString();
          parseSkipFile(patternsFileName);
        }
      }
}

How to fix it:
Change above highlighted line to
conf.getBoolean("wordcount.skip.patterns", false))


**************** CAUTION - Disclaimer *****************
This e-mail contains PRIVILEGED AND CONFIDENTIAL INFORMATION intended solely 
for the use of the addressee(s). If you are not the intended recipient, please 
notify the sender by e-mail and delete the original message. Further, you are not 
to copy, disclose, or distribute this e-mail or its contents to any other person and 
any such actions are unlawful. This e-mail may contain viruses. Infosys has taken 
every reasonable precaution to minimize this risk, but is not liable for any damage 
you may sustain as a result of any virus in this e-mail. You should carry out your 
own virus checks before opening the e-mail or attachment. Infosys reserves the 
right to monitor and review the content of all messages sent to or from this e-mail 
address. Messages sent to or from this e-mail address may be stored on the 
Infosys e-mail system.
***INFOSYS******** End of Disclaimer ********INFOSYS***

Mime
View raw message