hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dima Spivak <dspi...@cloudera.com>
Subject Re: Best way to pass configuration properties to MRv2 jobs
Date Mon, 18 Apr 2016 17:53:04 GMT
Probably better off asking on the Hadoop user mailing list (
user@hadoop.apache.org) than the HBase oneā€¦ :)

-Dima

On Mon, Apr 18, 2016 at 2:57 AM, Henning Blohm <henning.blohm@zfabrik.de>
wrote:

> Hi,
>
> in our Hadoop 2.6.0 cluster, we need to pass some properties to all Hadoop
> processes so they can be referenced using ${...} syntax in configuration
> files. This works reasonably well using HADOOP_NAMENODE_OPTS and the like.
>
> For Map/Reduce jobs however, we need to speficy not only
>
> mapred.child.java.opts
>
> to pass system properties, in addition we need to set
>
> yarn.app.mapreduce.am.command-opts
>
> for anything that is referenced in Hadoop configuration files.
>
> In the end however almost all the properties passed are available as
> environment variables as well.
>
> Hence my question:
>
> * Is it possible to use reference environment variables in configuration
> files directly?
> * Does anybody know of a simpler way to make sure some system properties
> are _always_ set for all Yarn processes?
>
> Thanks,
> Henning
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message