hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shahab Yunus <shahab.yu...@gmail.com>
Subject Re: How to access value of variable in Driver class which has been declared and modified inside Mapper class?
Date Tue, 12 May 2015 17:29:13 GMT
Better options than using static variable are, imo:

One option it use Counters. Check that API. We are using that for values
that are numeric and we need those in the driver once the job finishes. You
can create your custom counters too.

Other option is (if you need more than just one value or your info is
complicated) then we output data from each mapper or reducer as files in
each task (e.g. in the cleanup method) and then read those once the job

In both of these, you will have to collate and merge, according to the
logic all the copies of the value/info that you end up with. The number of
copies would be equal to the number of mappers or reducers (if you are
keeping the value there) used in the job.

You can also set the value in the job configuration and read that once the
is over.

On May 12, 2015 12:56 PM, "Answer Agrawal" <yrsna.tset01@gmail.com> wrote:

> Hi,
> I declared a variable and incremented/modified it inside Mapper class. Now
> I need to use the modified value of that variable in Driver class. I
> declared a static variable inside Mapper class and its modified value works
> in Driver class when I run the code in Eclipse IDE. But after creating that
> code as a runable jar from Eclipse and run jar file as “$ hadoop jar
> filename.jar input output” modified value does not reflect (value is 0) in
> Driver class.
> Thanks,

View raw message