hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sujit Pal <sujit....@comcast.net>
Subject how to pass values from mapper/reducer to driver code?
Date Thu, 24 Sep 2009 00:49:09 GMT
Hi,

I am using Hadoop 0.20. I can pass input values from the driver code
into the mapper/reducer code using the Configuration, something like
this:

// driver code:
Job job = new Job(conf, "name");
job.getConfiguration("foo", "bar");

// and in the mapper/reducer class, retrieve it like so:
String foo = context.getConfiguration().get("foo");

I am calculating some values from within my mapper as a side effect.
Regardless of which mapper drops it, it is going to be the same (ie it
is not affected by the split size). So I tried doing something like
this:

In the mapper's cleanup() method, I have:

context.getConfiguration().set("r", "result");

but when I try to retrieve it on the driver:
String result = job.getConfiguration().get("r");

I get a null in the result, so it appears as though I am either using
the wrong data structure and that 2-way communication is not supported.

Are there any other ways of getting at this sort of data? I read about
the Reporter, and I thought that perhaps I could use that, but not sure
how to do that.

Any pointers would be greatly appreciated.

TIA
Sujit



Mime
View raw message