hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: Pseudo distributed mode : How to increase no of concurrent map task
Date Sat, 29 Sep 2012 21:14:08 GMT

The right answer would be: Do not use config strings, use API points
available for each config tweak allowed at the client/job level.
However, things may not be complete here and there may be advanced
level of tuning that we didn't want to provide/support an API for. So…

With 2.x+, which renamed much of config files for consistency,
hopefully, you can rely on names. If a config name goes
<project>.<daemonname>.configname, such as "yarn.nodemanager.foo",
then it is daemon-specific. Otherwise, client overridable in one way
or the other. A certain level of this hint can be applied to
1.x/0.20.x as well, but there are certain property names that flout
that standard format from the past.

On Sun, Sep 30, 2012 at 2:34 AM, Jay Vyas <jayunit100@gmail.com> wrote:
> Hmmm...  I always make this mistake on my hadoop vm -- trying to set
> parameters which require xml settings in the conf.setInt(...) API at
> runtime, which sometimes has no effect.
> How can we know, (without having to individually troubleshoot a parameter)
> which parameters CAN versus CANNOT be set programmatically during a m/r job?

Harsh J

View raw message