hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gino Gu01 <Gino_G...@infosys.com>
Subject RE: An issue in MapReduce Tutorial
Date Mon, 24 Nov 2014 09:42:07 GMT
Adding  the link to the tutorial.

From: Gino Gu01
Sent: Monday, November 24, 2014 5:29 PM
To: 'user@hadoop.apache.org'
Subject: An issue in MapReduce Tutorial

There is one bug in WordCount v2.0 which is part of MapReduce Tutorial.
How to reproduce:

Run the application:

$ bin/hadoop jar wc.jar WordCount2 /user/joe/wordcount/input /user/joe/wordcount/output
It will throw Null Pointer Exception during map phase.

Below highlighted line set the default value of wordcount.skip.patterns to true.
But in the arguments we didn't pass the any patterns file, so the line for (URI patternsURI
: patternsURIs) throws exception.
    public void setup(Context context) throws IOException,
        InterruptedException {
      conf = context.getConfiguration();
      caseSensitive = conf.getBoolean("wordcount.case.sensitive", true);
      if (conf.getBoolean("wordcount.skip.patterns", true)) {
        URI[] patternsURIs = Job.getInstance(conf).getCacheFiles();
        for (URI patternsURI : patternsURIs) {
          Path patternsPath = new Path(patternsURI.getPath());
          String patternsFileName = patternsPath.getName().toString();

How to fix it:
Change above highlighted line to
conf.getBoolean("wordcount.skip.patterns", false))

**************** CAUTION - Disclaimer *****************
This e-mail contains PRIVILEGED AND CONFIDENTIAL INFORMATION intended solely 
for the use of the addressee(s). If you are not the intended recipient, please 
notify the sender by e-mail and delete the original message. Further, you are not 
to copy, disclose, or distribute this e-mail or its contents to any other person and 
any such actions are unlawful. This e-mail may contain viruses. Infosys has taken 
every reasonable precaution to minimize this risk, but is not liable for any damage 
you may sustain as a result of any virus in this e-mail. You should carry out your 
own virus checks before opening the e-mail or attachment. Infosys reserves the 
right to monitor and review the content of all messages sent to or from this e-mail 
address. Messages sent to or from this e-mail address may be stored on the 
Infosys e-mail system.
***INFOSYS******** End of Disclaimer ********INFOSYS***

View raw message