hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apurba Kumar Nath <apu...@pramati.com>
Subject smart serialization for WordCount example
Date Mon, 16 Nov 2009 07:10:15 GMT
I was wondering if we could introduce a template to let the end user add 
strings and ints to the context and the template could convert them to 
the appropriate serializable type. I am suggesting that the current
context.write(word, one); where word is Text and one is IntWritable, use 
something like context.write(itr.nextToken, 1); now either the context 
is smart enough to convert itr.nextToken which is string into Text and 1 
which is int into IntWritable or the map method is worked on by some 
method in the template to do the same. The advantage is that for 
beginners they can still talk in ints and strings and other such classes 
and may not have to get exposed to IntWritable and other such more 
easily serializable wrappers. I can commit the patch if the idea seems ok.


Share interests with friends with just a click. Try it!

View raw message