hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shahab Yunus <shahab.yu...@gmail.com>
Subject Re: How to access value of variable in Driver class which has been declared and modified inside Mapper class?
Date Tue, 12 May 2015 17:33:18 GMT
Here are some examples of how to use custom counters:
http://www.ashishpaliwal.com/blog/2012/05/hadoop-recipe-using-custom-java-counters/

Regards,
Shahab
On May 12, 2015 1:29 PM, "Shahab Yunus" <shahab.yunus@gmail.com> wrote:

> Better options than using static variable are, imo:
>
> One option it use Counters. Check that API. We are using that for values
> that are numeric and we need those in the driver once the job finishes. You
> can create your custom counters too.
>
> Other option is (if you need more than just one value or your info is
> complicated) then we output data from each mapper or reducer as files in
> each task (e.g. in the cleanup method) and then read those once the job
> finishes.
>
> In both of these, you will have to collate and merge, according to the
> logic all the copies of the value/info that you end up with. The number of
> copies would be equal to the number of mappers or reducers (if you are
> keeping the value there) used in the job.
>
> You can also set the value in the job configuration and read that once the
> is over.
>
> Regards,
> Shahab
> On May 12, 2015 12:56 PM, "Answer Agrawal" <yrsna.tset01@gmail.com> wrote:
>
>> Hi,
>>
>> I declared a variable and incremented/modified it inside Mapper class.
>> Now I need to use the modified value of that variable in Driver class. I
>> declared a static variable inside Mapper class and its modified value works
>> in Driver class when I run the code in Eclipse IDE. But after creating that
>> code as a runable jar from Eclipse and run jar file as “$ hadoop jar
>> filename.jar input output” modified value does not reflect (value is 0) in
>> Driver class.
>>
>> Thanks,
>>
>

Mime
View raw message