hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Owen O'Malley <omal...@apache.org>
Subject Re: writable class to be used to read floating point values from input?
Date Sun, 26 Oct 2008 05:24:43 GMT

On Oct 25, 2008, at 8:32 PM, pols cut wrote:

> I am trying to write a map reduce function which takes take the  
> following types of <key,value> pairs
> Map function -- should read floating point values (i dont really  
> care about  key)
> it should output <null,floatwritable>

If the input is stored in a text file, using TextInputFormat is right.  
Your map inputs will be:

LongWritable, Text

Just use the Text and convert it to a Double.

> reduce -- input- <null,floatwritable>
>                output <null,floatwritable>

This doesn't make any sense. How should the input to the reduce be  
sorted? By the float? In that case, it would be:

FloatWritable, NullWritable

You will get one call to the reduce for each distinct float value the  
maps generate. The reduce can iterate through the NullWritables to see  
how many times that key was generated.

-- Owen

View raw message