hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From himanshu chandola <himanshu_cool...@yahoo.com>
Subject questions on map and reduce
Date Sun, 06 Sep 2009 19:52:49 GMT
Hi Everyone,
What would be the best way to get map to output values  in different formats (reduce needs
to add the integer ones) . I realize that writing an ObjectWritable (not of the primitive
type that comes with hadoop) might be the best. I basically am worried that if I output integers
to Text and convert them back to IntWritable, it would be a performance overhead. Would it
be significant enough to worry about ? And can you think of anything other than ObjectWritable
or having everything as Text to do this ?

And another question was is there a way to get reduce to set values in JobConf or do I just
flush that into disk and let the sequential job to read that. Basically , am I reading too
much into the performance overhead thing ?



Thanks

 Morpheus: Do you believe in fate, Neo?
Neo: No.
Morpheus: Why Not?
Neo: Because I don't like the idea that I'm not in control of my life.



      

Mime
View raw message