Mailing-List: contact general-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: general@hadoop.apache.org
Received-SPF: pass (nike.apache.org: local policy)
Message-ID: <4BFE3004.5000603@echo.fr>
Date: Thu, 27 May 2010 10:40:36 +0200
From: Varene Olivier <varene@echo.fr>
User-Agent: Thunderbird 2.0.0.23 (X11/20090817)
MIME-Version: 1.0
To: general@hadoop.apache.org
Subject: Mapper Reducer : Unit Test and mocking with static variables
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Hello to all,

first of all many thanks for this great piece of software you are all
contributing to :)

I am actually creating a program in Hadoop MapReduce, but I intend to
massively use JUnit Tests to validate my code.

To test a Mapper, I mock a Mapper.Context object and run the Mapper.map
function with it, but inside my Mapper.map function, I am using Counters 
to maintain internal statistics. The update of those counters make the 
test fails with a null pointer (JavaNullPointerException).
There is an issue "Counters" of variable visibility from my test ...

Do you have any idea how to mock/validate/overcome this issue ?


Thanks a lots
Best Regards
Olivier Varene


: Code :
--------

private static enum Counters {NBLINES};
private static IntWritable one = new IntWritable(1);
private static LongWritable oneL = new LongWritable(1);


// Mapper
/**
  * outputs 1 for each line encountered
  */
public static class LCMapper extends
	Mapper<LongWritable, Text, IntWritable, LongWritable>
{
   private final static IntWritable one = new IntWritable(1);
   private final static LongWritable oneL = new LongWritable(1);
   /**
   * for each line encountered outputs 1 for the key 1
   */
   public void map(LongWritable p_key, Text p_inputs, Context p_context)
	throws IOException, InterruptedException
   {
     p_context.getCounter(Counters.NBLINES).increment(1);
     p_context.write(one,oneL);	
   } // end map
} // end Mapper

: Test :
--------

@Test
public void testLCMapper() throws IOException, InterruptedException
{	
   LCMapper mapper = new LCMapper();
   Text value = new Text("ligneA\nlignB\n");
   // mock the map job execution context
   Context context = mock(Context.class);
   try {
     mapper.map(null, value, context);
   } catch (NullPointerException e)
   {
     // exception raised by context.getCounter(...) null pointer
     // HOW TO MOCK ???
     // Counter is a static enum from the englobing class ...
     // issue with variable visibility (from map function) in this test
   }
   // verify that the Mapper issued (one,oneL) pair
   verify(context).write(one,oneL);
		
} // end testLCMapper