ambari-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mark S (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AMBARI-15273) Cannot set ${hadoop.tmp.dir}
Date Wed, 24 Aug 2016 16:51:20 GMT

    [ https://issues.apache.org/jira/browse/AMBARI-15273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15435273#comment-15435273
] 

Mark S commented on AMBARI-15273:
---------------------------------

Just to follow up.

You can configure the $\{hadoop.tmp.dir\} property via the web interface http://localhost:8080/#/main/services/HDFS/configs,
and add the entry to "Custom core-site" via the "Add Property" link.

I was able to verify this property being set correctly with the following command:

{code}
 sudo -u hdfs /usr/hdp/current/hadoop-client/bin/hadoop org.apache.hadoop.conf.Configuration
| grep hadoop.tmp.dir

<property><name>hadoop.tmp.dir</name><value>/data/hadoop-tmp/${user.name}</value><source>core-site.xml</source></property>
<property><name>fs.s3.buffer.dir</name><value>${hadoop.tmp.dir}/s3</value><source>core-default.xml</source></property>
<property><name>fs.s3a.buffer.dir</name><value>${hadoop.tmp.dir}/s3a</value><source>core-default.xml</source></property>
<property><name>io.seqfile.local.dir</name><value>${hadoop.tmp.dir}/io/local</value><source>core-default.xml</source></property>
{code}


> Cannot set ${hadoop.tmp.dir}
> ----------------------------
>
>                 Key: AMBARI-15273
>                 URL: https://issues.apache.org/jira/browse/AMBARI-15273
>             Project: Ambari
>          Issue Type: Bug
>    Affects Versions: 2.1.2
>         Environment: CentOS Linux release 7.2.1511 (Core)
> ambari-agent-2.1.2-377.x86_64
> ambari-server-2.1.2-377.x86_64
> ambari-metrics-hadoop-sink-2.1.2-377.x86_64
> ambari-metrics-monitor-2.1.2-377.x86_64
> ambari-metrics-collector-2.1.2-377.x86_64
> HDFS 	2.7.1.2.3
> MapReduce2 	2.7.1.2.3
> YARN 	2.7.1.2.3
>            Reporter: Mark S
>
> I am trying to set the "hadoop.tmp.dir" property, as a work-around for a situation where
my Linux "/tmp" partition is running out of space and causing my Hadoop MapReduce jobs to
fail.
> Currently the following folders are causing my tmp partition to fill up and my Hadoop
Map Reduce to fail,
> {code}
> /tmp/hadoop-user/mapred/local
> /tmp/hadoop-user/mapred/staging
> {code}
> and I am pretty sure is set by the following two properties:
> {code}
> # https://hadoop.apache.org/docs/r2.7.1/hadoop-mapreduce-client/hadoop-mapreduce-client-core/mapred-default.xml
> mapreduce.cluster.local.dir=${hadoop.tmp.dir}/mapred/local
> mapreduce.jobtracker.staging.root.dir=${hadoop.tmp.dir}/mapred/staging
> {code}
> Thus far, I have been trying unsuccessfully to set the "hadoop.tmp.dir" property using
the following command and restarting all my services after this.
> {code}
> # https://cwiki.apache.org/confluence/display/AMBARI/Modify+configurations
> /var/lib/ambari-server/resources/scripts/configs.sh -u admin -p admin set <ambari_host>
<ambari_cluster_name> "core-site" "hadoop.tmp.dir" "/data/hadoop-tmp/\${user.name}"
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message