hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mori Bellamy <mbell...@apple.com>
Subject hadoop reduce : very weird behavior
Date Mon, 16 Jun 2008 23:50:38 GMT
hey all,
i searched the archives for this issue, but didn't find anything (but  
apologies in advanced if i missed it).

i'm running into some strange behavior with a mapreduce operation. the  
mapreduce function has the type of MyFunction: Long/Text -> Text/Int - 
 > Text/Float. the first pair is my recordreader (its the default type  
for reading files line by line). the last two pairs are specific to my  
task: each key goes to one number. i know ahead of time that my input  
will have every key exactly twice. i'm interested in outputting the  
ratio of each value for a given key. so for instance, lets say my  
input is

foo	1
bar	2
foo	3
bar	4

i want this as output:

foo	.33333333
bar	.5

so in my reduce function, i do something like
first = iter.next.get()
second = iter.next.get()

and then proceed to compute the ratios. but my code still "works" on  
the following input!

foobar	1
foobas	2

output: foobar	.5

does anybody know the cause of this?


View raw message