hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: Please help with hadoop configuration parameter set and get
Date Fri, 17 Dec 2010 14:39:12 GMT
You can use hadoop counter to pass this information.
This way, you see the counters in job report.

On Thu, Dec 16, 2010 at 10:58 PM, Peng, Wei <Wei.Peng@xerox.com> wrote:

> Hi,
>
>
>
> I am a newbie of hadoop.
>
> Today I was struggling with a hadoop problem for several hours.
>
>
>
> I initialize a parameter by setting job configuration in main.
>
> E.g. Configuration con = new Configuration();
>
> con.set("test", "1");
>
> Job job = new Job(con);
>
>
>
> Then in the mapper class, I want to set "test" to "2". I did it by
>
> context.getConfiguration().set("test","2");
>
>
>
> Finally in the main method, after the job is finished, I check the
> "test" again by
>
> job.getConfiguration().get("test");
>
>
>
> However, the value of "test" is still "1".
>
>
>
> The reason why I want to change the parameter inside Mapper class is
> that I want to determine when to stop an iteration in the main method.
> For example, for doing breadth-first search, when there is no new nodes
> are added for further expansion, the searching iteration should stop.
>
>
>
> Your help will be deeply appreciated. Thank you
>
>
>
> Wei
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message