hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Himanshu <ll_oz...@yahoo.com.hk>
Subject sharing variables across chained jobs
Date Wed, 23 Dec 2009 07:51:05 GMT
Hi everyone,
I run multiple map/reduce jobs which are
chained together. The output of one map/reduce is the input of another.
There are also some integer valued variables which are outputted from
one map/reduce job and used as input in the consequent one. These
variables are got by summing up integer valued data having a certain
key in the reduce step. 

My question was - what would be the
best way to share these several integer valued variables across
multiply map/reduce iterations? I don't want to write each of them to a
separate file in the reduce step and consequently read those files in
the next iteration of map/reduce. 

Looking forward to all the suggestions


Send instant messages to your online friends http://uk.messenger.yahoo.com 

View raw message