incubator-ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Siddharth Wagle (JIRA)" <j...@apache.org>
Subject [jira] [Created] (AMBARI-3074) Ambari wont start NodeManager because one of multiple folders not created
Date Tue, 03 Sep 2013 16:14:51 GMT
Siddharth Wagle created AMBARI-3074:
---------------------------------------

             Summary: Ambari wont start NodeManager because one of multiple folders not created
                 Key: AMBARI-3074
                 URL: https://issues.apache.org/jira/browse/AMBARI-3074
             Project: Ambari
          Issue Type: Bug
          Components: agent
    Affects Versions: 1.4.0
            Reporter: Siddharth Wagle
            Assignee: Siddharth Wagle
             Fix For: 1.4.1


{{yarn-site}} having:
{noformat}
"yarn.nodemanager.local-dirs" : "/grid/0/hadoop/yarn,/grid/1/hadoop/yarn,/grid/2/hadoop/yarn,/grid/3/hadoop/yarn,/grid/4/hadoop/yarn,/grid/5/hadoop/yarn",
"yarn.nodemanager.log-dirs" : "/grid/0/hadoop/yarn,/grid/1/hadoop/yarn,/grid/2/hadoop/yarn,/grid/3/hadoop/yarn,/grid/4/hadoop/yarn,/grid/5/hadoop/yarn",
{noformat}

Now {{/grid/3}} was mounted as read-only due to some disk errors. Though other folders got
successfully created, Ambari will not start the NodeManager process.
{noformat}
notice: /Stage[1]/Hdp::Snappy::Package/Hdp::Snappy::Package::Ln[32]/Hdp::Exec[hdp::snappy::package::ln
32]/Exec[hdp::snappy::package::ln 32]/returns: executed successfully
notice: /Stage[2]/Hdp-yarn::Nodemanager/Hdp-yarn::Nodemanager::Create_nm_dirs[/grid/3/hadoop/yarn]/Hdp::Directory_recursive_create[/grid/3/hadoop/yarn]/Hdp::Exec[mkdir
-p /grid/3/hadoop/yarn]/Exec[mkdir -p /grid/3/hadoop/yarn]/returns: mkdir: cannot create directory
`/grid/3/hadoop': Read-only file system
err: /Stage[2]/Hdp-yarn::Nodemanager/Hdp-yarn::Nodemanager::Create_nm_dirs[/grid/3/hadoop/yarn]/Hdp::Directory_recursive_create[/grid/3/hadoop/yarn]/Hdp::Exec[mkdir
-p /grid/3/hadoop/yarn]/Exec[mkdir -p /grid/3/hadoop/yarn]/returns: change from notrun to
0 failed: mkdir -p /grid/3/hadoop/yarn returned 1 instead of one of [0] at /var/lib/ambari-agent/puppet/modules/hdp/manifests/init.pp:479
notice: /Stage[2]/Hdp-yarn::Nodemanager/Hdp-yarn::Nodemanager::Create_nm_dirs[/grid/3/hadoop/yarn]/Hdp::Directory_recursive_create[/grid/3/hadoop/yarn]/Hdp::Exec[mkdir
-p /grid/3/hadoop/yarn]/Anchor[hdp::exec::mkdir -p /grid/3/hadoop/yarn::end]: Dependency Exec[mkdir
-p /grid/3/hadoop/yarn] has failures: true
warning: /Stage[2]/Hdp-yarn::Nodemanager/Hdp-yarn::Nodemanager::Create_nm_dirs[/grid/3/hadoop/yarn]/Hdp::Directory_recursive_create[/grid/3/hadoop/yarn]/Hdp::Exec[mkdir
-p /grid/3/hadoop/yarn]/Anchor[hdp::exec::mkdir -p /grid/3/hadoop/yarn::end]: Skipping because
of failed dependencies
notice: /Stage[2]/Hdp-yarn::Nodemanager/Hdp-yarn::Nodemanager::Create_nm_dirs[/grid/3/hadoop/yarn]/Hdp::Directory_recursive_create[/grid/3/hadoop/yarn]/Hdp::Directory[/grid/3/hadoop/yarn]/File[/grid/3/hadoop/yarn]:
Dependency Exec[mkdir -p /grid/3/hadoop/yarn] has failures: true
warning: /Stage[2]/Hdp-yarn::Nodemanager/Hdp-yarn::Nodemanager::Create_nm_dirs[/grid/3/hadoop/yarn]/Hdp::Directory_recursive_create[/grid/3/hadoop/yarn]/Hdp::Directory[/grid/3/hadoop/yarn]/File[/grid/3/hadoop/yarn]:
Skipping because of failed dependencies
notice: /Stage[2]/Hdp-yarn::Initialize/Hdp-yarn::Generate_common_configs[yarn-common-configs]/Configgenerator::Configfile[capacity-scheduler]/File[/etc/hadoop/conf/capacity-scheduler.xml]/content:
content changed '{md5}e5d17c21c7a5e1db9f3af35cba71df0a' to '{md5}2ca1d267a46f1aecac726caabaa16774'
notice: /Stage[2]/Hdp-yarn::Initialize/Hdp-yarn::Generate_common_configs[yarn-common-configs]/Configgenerator::Configfile[capacity-scheduler]/File[/etc/hadoop/conf/capacity-scheduler.xml]/owner:
owner changed 'hdfs' to 'yarn'
notice: /Stage[2]/Hdp-yarn::Initialize/Hdp-yarn::Generate_common_configs[yarn-common-configs]/Configgenerator::Configfile[core-site]/File[/etc/hadoop/conf/core-site.xml]/content:
content changed '{md5}86d742a780d59a957ea0a283dec03784' to '{md5}8506e4402ba8140ea4f9fed97b6f94e2'
notice: /Stage[2]/Hdp-yarn::Initialize/Hdp-yarn::Generate_common_configs[yarn-common-configs]/Configgenerator::Configfile[yarn-site]/File[/etc/hadoop/conf/yarn-site.xml]/content:
content changed '{md5}d84a967ce47a6b77734ed8f53d817c6e' to '{md5}42940cca6e8f64ae5de50524fb131274'
notice: /Stage[2]/Hdp-yarn::Nodemanager/Hdp-yarn::Service[nodemanager]/Anchor[hdp-yarn::service::nodemanager::begin]:
Dependency Exec[mkdir -p /grid/3/hadoop/yarn] has failures: true
warning: /Stage[2]/Hdp-yarn::Nodemanager/Hdp-yarn::Service[nodemanager]/Anchor[hdp-yarn::service::nodemanager::begin]:
Skipping because of failed dependencies
notice: /Stage[2]/Hdp-yarn::Nodemanager/Hdp-yarn::Service[nodemanager]/Hdp::Directory_recursive_create[/var/log/hadoop-yarn]/Hdp::Exec[mkdir
-p /var/log/hadoop-yarn]/Anchor[hdp::exec::mkdir -p /var/log/hadoop-yarn::begin]: Dependency
Exec[mkdir -p /grid/3/hadoop/yarn] has failures: true
warning: /Stage[2]/Hdp-yarn::Nodemanager/Hdp-yarn::Service[nodemanager]/Hdp::Directory_recursive_create[/var/log/hadoop-yarn]/Hdp::Exec[mkdir
-p /var/log/hadoop-yarn]/Anchor[hdp::exec::mkdir -p /var/log/hadoop-yarn::begin]: Skipping
because of failed dependencies
notice: /Stage[2]/Hdp-yarn::Nodemanager/Hdp-yarn::Service[nodemanager]/Hdp::Directory_recursive_create[/var/log/hadoop-yarn]/Hdp::Exec[mkdir
-p /var/log/hadoop-yarn]/Exec[mkdir -p /var/log/hadoop-yarn]: Dependency Exec[mkdir -p /grid/3/hadoop/yarn]
has failures: true
warning: /Stage[2]/Hdp-yarn::Nodemanager/Hdp-yarn::Service[nodemanager]/Hdp::Directory_recursive_create[/var/log/hadoop-yarn]/Hdp::Exec[mkdir
-p /var/log/hadoop-yarn]/Exec[mkdir -p /var/log/hadoop-yarn]: Skipping because of failed dependencies
notice: /Stage[2]/Hdp-yarn::Nodemanager/Hdp-yarn::Service[nodemanager]/Hdp::Directory_recursive_create[/var/log/hadoop-yarn]/Hdp::Exec[mkdir
-p /var/log/hadoop-yarn]/Anchor[hdp::exec::mkdir -p /var/log/hadoop-yarn::end]: Dependency
Exec[mkdir -p /grid/3/hadoop/yarn] has failures: true
warning: /Stage[2]/Hdp-yarn::Nodemanager/Hdp-yarn::Service[nodemanager]/Hdp::Directory_recursive_create[/var/log/hadoop-yarn]/Hdp::Exec[mkdir
-p /var/log/hadoop-yarn]/Anchor[hdp::exec::mkdir -p /var/log/hadoop-yarn::end]: Skipping because
of failed dependencies
notice: /Stage[2]/Hdp-yarn::Nodemanager/Hdp-yarn::Service[nodemanager]/Hdp::Directory_recursive_create[/var/log/hadoop-yarn]/Hdp::Directory[/var/log/hadoop-yarn]/File[/var/log/hadoop-yarn]:
Dependency Exec[mkdir -p /grid/3/hadoop/yarn] has failures: true
warning: /Stage[2]/Hdp-yarn::Nodemanager/Hdp-yarn::Service[nodemanager]/Hdp::Directory_recursive_create[/var/log/hadoop-yarn]/Hdp::Directory[/var/log/hadoop-yarn]/File[/var/log/hadoop-yarn]:
Skipping because of failed dependencies
notice: /Stage[2]/Hdp-yarn::Nodemanager/Hdp-yarn::Service[nodemanager]/Hdp::Directory_recursive_create[/var/run/hadoop-yarn/yarn]/Hdp::Exec[mkdir
-p /var/run/hadoop-yarn/yarn]/Anchor[hdp::exec::mkdir -p /var/run/hadoop-yarn/yarn::begin]:
Dependency Exec[mkdir -p /grid/3/hadoop/yarn] has failures: true
warning: /Stage[2]/Hdp-yarn::Nodemanager/Hdp-yarn::Service[nodemanager]/Hdp::Directory_recursive_create[/var/run/hadoop-yarn/yarn]/Hdp::Exec[mkdir
-p /var/run/hadoop-yarn/yarn]/Anchor[hdp::exec::mkdir -p /var/run/hadoop-yarn/yarn::begin]:
Skipping because of failed dependencies
notice: /Stage[2]/Hdp-yarn::Nodemanager/Hdp-yarn::Service[nodemanager]/Hdp::Directory_recursive_create[/var/run/hadoop-yarn/yarn]/Hdp::Exec[mkdir
-p /var/run/hadoop-yarn/yarn]/Exec[mkdir -p /var/run/hadoop-yarn/yarn]: Dependency Exec[mkdir
-p /grid/3/hadoop/yarn] has failures: true
warning: /Stage[2]/Hdp-yarn::Nodemanager/Hdp-yarn::Service[nodemanager]/Hdp::Directory_recursive_create[/var/run/hadoop-yarn/yarn]/Hdp::Exec[mkdir
-p /var/run/hadoop-yarn/yarn]/Exec[mkdir -p /var/run/hadoop-yarn/yarn]: Skipping because of
failed dependencies
notice: /Stage[2]/Hdp-yarn::Nodemanager/Hdp-yarn::Service[nodemanager]/Hdp::Directory_recursive_create[/var/run/hadoop-yarn/yarn]/Hdp::Exec[mkdir
-p /var/run/hadoop-yarn/yarn]/Anchor[hdp::exec::mkdir -p /var/run/hadoop-yarn/yarn::end]:
Dependency Exec[mkdir -p /grid/3/hadoop/yarn] has failures: true
warning: /Stage[2]/Hdp-yarn::Nodemanager/Hdp-yarn::Service[nodemanager]/Hdp::Directory_recursive_create[/var/run/hadoop-yarn/yarn]/Hdp::Exec[mkdir
-p /var/run/hadoop-yarn/yarn]/Anchor[hdp::exec::mkdir -p /var/run/hadoop-yarn/yarn::end]:
Skipping because of failed dependencies
notice: /Stage[2]/Hdp-yarn::Nodemanager/Hdp-yarn::Service[nodemanager]/Hdp::Directory_recursive_create[/var/run/hadoop-yarn/yarn]/Hdp::Directory[/var/run/hadoop-yarn/yarn]/File[/var/run/hadoop-yarn/yarn]:
Dependency Exec[mkdir -p /grid/3/hadoop/yarn] has failures: true
warning: /Stage[2]/Hdp-yarn::Nodemanager/Hdp-yarn::Service[nodemanager]/Hdp::Directory_recursive_create[/var/run/hadoop-yarn/yarn]/Hdp::Directory[/var/run/hadoop-yarn/yarn]/File[/var/run/hadoop-yarn/yarn]:
Skipping because of failed dependencies
notice: /Stage[2]/Hdp-yarn::Nodemanager/Hdp-yarn::Service[nodemanager]/Hdp::Exec[su - yarn
-c  'export HADOOP_LIBEXEC_DIR=/usr/lib/hadoop/libexec && /usr/lib/hadoop-yarn/sbin/yarn-daemon.sh
--config /etc/hadoop/conf start nodemanager']/Anchor[hdp::exec::su - yarn -c  'export HADOOP_LIBEXEC_DIR=/usr/lib/hadoop/libexec
&& /usr/lib/hadoop-yarn/sbin/yarn-daemon.sh --config /etc/hadoop/conf start nodemanager'::begin]:
Dependency Exec[mkdir -p /grid/3/hadoop/yarn] has failures: true
warning: /Stage[2]/Hdp-yarn::Nodemanager/Hdp-yarn::Service[nodemanager]/Hdp::Exec[su - yarn
-c  'export HADOOP_LIBEXEC_DIR=/usr/lib/hadoop/libexec && /usr/lib/hadoop-yarn/sbin/yarn-daemon.sh
--config /etc/hadoop/conf start nodemanager']/Anchor[hdp::exec::su - yarn -c  'export HADOOP_LIBEXEC_DIR=/usr/lib/hadoop/libexec
&& /usr/lib/hadoop-yarn/sbin/yarn-daemon.sh --config /etc/hadoop/conf start nodemanager'::begin]:
Skipping because of failed dependencies
notice: /Stage[2]/Hdp-yarn::Nodemanager/Hdp-yarn::Service[nodemanager]/Hdp::Exec[su - yarn
-c  'export HADOOP_LIBEXEC_DIR=/usr/lib/hadoop/libexec && /usr/lib/hadoop-yarn/sbin/yarn-daemon.sh
--config /etc/hadoop/conf start nodemanager']/Exec[su - yarn -c  'export HADOOP_LIBEXEC_DIR=/usr/lib/hadoop/libexec
&& /usr/lib/hadoop-yarn/sbin/yarn-daemon.sh --config /etc/hadoop/conf start nodemanager']:
Dependency Exec[mkdir -p /grid/3/hadoop/yarn] has failures: true
warning: /Stage[2]/Hdp-yarn::Nodemanager/Hdp-yarn::Service[nodemanager]/Hdp::Exec[su - yarn
-c  'export HADOOP_LIBEXEC_DIR=/usr/lib/hadoop/libexec && /usr/lib/hadoop-yarn/sbin/yarn-daemon.sh
--config /etc/hadoop/conf start nodemanager']/Exec[su - yarn -c  'export HADOOP_LIBEXEC_DIR=/usr/lib/hadoop/libexec
&& /usr/lib/hadoop-yarn/sbin/yarn-daemon.sh --config /etc/hadoop/conf start nodemanager']:
Skipping because of failed dependencies
notice: /Stage[2]/Hdp-yarn::Nodemanager/Hdp-yarn::Service[nodemanager]/Hdp::Exec[su - yarn
-c  'export HADOOP_LIBEXEC_DIR=/usr/lib/hadoop/libexec && /usr/lib/hadoop-yarn/sbin/yarn-daemon.sh
--config /etc/hadoop/conf start nodemanager']/Anchor[hdp::exec::su - yarn -c  'export HADOOP_LIBEXEC_DIR=/usr/lib/hadoop/libexec
&& /usr/lib/hadoop-yarn/sbin/yarn-daemon.sh --config /etc/hadoop/conf start nodemanager'::end]:
Dependency Exec[mkdir -p /grid/3/hadoop/yarn] has failures: true
warning: /Stage[2]/Hdp-yarn::Nodemanager/Hdp-yarn::Service[nodemanager]/Hdp::Exec[su - yarn
-c  'export HADOOP_LIBEXEC_DIR=/usr/lib/hadoop/libexec && /usr/lib/hadoop-yarn/sbin/yarn-daemon.sh
--config /etc/hadoop/conf start nodemanager']/Anchor[hdp::exec::su - yarn -c  'export HADOOP_LIBEXEC_DIR=/usr/lib/hadoop/libexec
&& /usr/lib/hadoop-yarn/sbin/yarn-daemon.sh --config /etc/hadoop/conf start nodemanager'::end]:
Skipping because of failed dependencies
notice: /Stage[2]/Hdp-yarn::Nodemanager/Hdp-yarn::Service[nodemanager]/Hdp::Exec[sleep 5;
ls /var/run/hadoop-yarn/yarn/yarn-yarn-nodemanager.pid >/dev/null 2>&1 &&
ps `cat /var/run/hadoop-yarn/yarn/yarn-yarn-nodemanager.pid` >/dev/null 2>&1]/Anchor[hdp::exec::sleep
5; ls /var/run/hadoop-yarn/yarn/yarn-yarn-nodemanager.pid >/dev/null 2>&1 &&
ps `cat /var/run/hadoop-yarn/yarn/yarn-yarn-nodemanager.pid` >/dev/null 2>&1::begin]:
Dependency Exec[mkdir -p /grid/3/hadoop/yarn] has failures: true
warning: /Stage[2]/Hdp-yarn::Nodemanager/Hdp-yarn::Service[nodemanager]/Hdp::Exec[sleep 5;
ls /var/run/hadoop-yarn/yarn/yarn-yarn-nodemanager.pid >/dev/null 2>&1 &&
ps `cat /var/run/hadoop-yarn/yarn/yarn-yarn-nodemanager.pid` >/dev/null 2>&1]/Anchor[hdp::exec::sleep
5; ls /var/run/hadoop-yarn/yarn/yarn-yarn-nodemanager.pid >/dev/null 2>&1 &&
ps `cat /var/run/hadoop-yarn/yarn/yarn-yarn-nodemanager.pid` >/dev/null 2>&1::begin]:
Skipping because of failed dependencies
notice: /Stage[2]/Hdp-yarn::Nodemanager/Hdp-yarn::Service[nodemanager]/Hdp::Exec[sleep 5;
ls /var/run/hadoop-yarn/yarn/yarn-yarn-nodemanager.pid >/dev/null 2>&1 &&
ps `cat /var/run/hadoop-yarn/yarn/yarn-yarn-nodemanager.pid` >/dev/null 2>&1]/Exec[sleep
5; ls /var/run/hadoop-yarn/yarn/yarn-yarn-nodemanager.pid >/dev/null 2>&1 &&
ps `cat /var/run/hadoop-yarn/yarn/yarn-yarn-nodemanager.pid` >/dev/null 2>&1]: Dependency
Exec[mkdir -p /grid/3/hadoop/yarn] has failures: true
warning: /Stage[2]/Hdp-yarn::Nodemanager/Hdp-yarn::Service[nodemanager]/Hdp::Exec[sleep 5;
ls /var/run/hadoop-yarn/yarn/yarn-yarn-nodemanager.pid >/dev/null 2>&1 &&
ps `cat /var/run/hadoop-yarn/yarn/yarn-yarn-nodemanager.pid` >/dev/null 2>&1]/Exec[sleep
5; ls /var/run/hadoop-yarn/yarn/yarn-yarn-nodemanager.pid >/dev/null 2>&1 &&
ps `cat /var/run/hadoop-yarn/yarn/yarn-yarn-nodemanager.pid` >/dev/null 2>&1]: Skipping
because of failed dependencies
notice: /Stage[2]/Hdp-yarn::Nodemanager/Hdp-yarn::Service[nodemanager]/Hdp::Exec[sleep 5;
ls /var/run/hadoop-yarn/yarn/yarn-yarn-nodemanager.pid >/dev/null 2>&1 &&
ps `cat /var/run/hadoop-yarn/yarn/yarn-yarn-nodemanager.pid` >/dev/null 2>&1]/Anchor[hdp::exec::sleep
5; ls /var/run/hadoop-yarn/yarn/yarn-yarn-nodemanager.pid >/dev/null 2>&1 &&
ps `cat /var/run/hadoop-yarn/yarn/yarn-yarn-nodemanager.pid` >/dev/null 2>&1::end]:
Dependency Exec[mkdir -p /grid/3/hadoop/yarn] has failures: true
warning: /Stage[2]/Hdp-yarn::Nodemanager/Hdp-yarn::Service[nodemanager]/Hdp::Exec[sleep 5;
ls /var/run/hadoop-yarn/yarn/yarn-yarn-nodemanager.pid >/dev/null 2>&1 &&
ps `cat /var/run/hadoop-yarn/yarn/yarn-yarn-nodemanager.pid` >/dev/null 2>&1]/Anchor[hdp::exec::sleep
5; ls /var/run/hadoop-yarn/yarn/yarn-yarn-nodemanager.pid >/dev/null 2>&1 &&
ps `cat /var/run/hadoop-yarn/yarn/yarn-yarn-nodemanager.pid` >/dev/null 2>&1::end]:
Skipping because of failed dependencies
notice: /Stage[2]/Hdp-yarn::Nodemanager/Hdp-yarn::Service[nodemanager]/Anchor[hdp-yarn::service::nodemanager::end]:
Dependency Exec[mkdir -p /grid/3/hadoop/yarn] has failures: true
warning: /Stage[2]/Hdp-yarn::Nodemanager/Hdp-yarn::Service[nodemanager]/Anchor[hdp-yarn::service::nodemanager::end]:
Skipping because of failed dependencies
notice: /Stage[2]/Hdp-yarn::Nodemanager/Anchor[hdp-yarn::nodemanager::end]: Dependency Exec[mkdir
-p /grid/3/hadoop/yarn] has failures: true
warning: /Stage[2]/Hdp-yarn::Nodemanager/Anchor[hdp-yarn::nodemanager::end]: Skipping because
of failed dependencies
notice: /Stage[2]/Hdp-yarn::Initialize/Hdp-yarn::Generate_common_configs[yarn-common-configs]/Configgenerator::Configfile[mapred-site]/File[/etc/hadoop/conf/mapred-site.xml]/content:
content changed '{md5}093cb1899b3c3b9dc4a7c1c93729c18b' to '{md5}4c462999cc47e6f6ba0e6381d71d81ba'
notice: /Stage[2]/Hdp-yarn::Initialize/Hdp-yarn::Generate_common_configs[yarn-common-configs]/Configgenerator::Configfile[mapred-site]/File[/etc/hadoop/conf/mapred-site.xml]/owner:
owner changed 'mapred' to 'yarn'
notice: Finished catalog run in 2.39 seconds
{noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message