ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Onischuk" <aonis...@hortonworks.com>
Subject Review Request 40581: During Component Install conf-select Shell Uses Current Working Directory That Doesn't Exist Yet
Date Sun, 22 Nov 2015 16:01:49 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40581/
-----------------------------------------------------------

Review request for Ambari and Dmitro Lisnichenko.


Bugs: AMBARI-14006
    https://issues.apache.org/jira/browse/AMBARI-14006


Repository: ambari


Description
-------

On an existing (not upgraded) HDP 2.3.2.0-2390 cluster, installation of
additional components fails with the following error:

    
    
    
    2015-10-21 08:28:44,613 - Skipping Execute[('cp', '-R', '-p', '/etc/zookeeper/conf', '/etc/zookeeper/conf.install')]
due to not_if 
    2015-10-21 08:28:44,767 - New conf directories: shell-init: error retrieving current directory:
getcwd: cannot access parent directories: No such file or directory, shell-init: error retrieving
current directory: getcwd: cannot access parent directories: No such file or directory, job-working-directory:
error retrieving current directory: getcwd: cannot access parent directories: No such file
or directory, /etc/zookeeper/2.3.2.0-2950/0 
    2015-10-21 08:28:45,075 - checked_call['conf-select set-conf-dir --package zookeeper --stack-version
2.3.2.0-2950 --conf-version 0'] {'logoutput': False, 'sudo': True, 'quiet': False} 
    2015-10-21 08:28:45,228 - checked_call returned (0, 'shell-init: error retrieving current
directory: getcwd: cannot access parent directories: No such file or directory\nshell-init:
error retrieving current directory: getcwd: cannot access parent directories: No such file
or directory\njob-working-directory: error retrieving current directory: getcwd: cannot access
parent directories: No such file or directory\n/usr/hdp/2.3.2.0-2950/zookeeper/conf ->
/etc/zookeeper/2.3.2.0-2950/0') 
    

Subsequent runs of the install command succeed. The root of the problem is:

    
    
    
    getcwd: cannot access parent directories: No such file or directory\n/usr/hdp/2.3.2.0-2950/zookeeper/conf
-> /etc/zookeeper/2.3.2.0-2950/0') 
    

During a component install, we invoke the `conf-select` module which:

  * Creates `conf.install` from `conf`
  * Changes `conf` into a symlink
  * Invoke `conf-select` to change pointers to <version>/0

However, the /etc/zookeeper/2.3.2.0-2950/0 does not yet exist while `conf-
select` is being invoked:

**after-INSTALL/scripts/shared_initialization.py**
    
    
    
    versioned_confs = conf_select.create("HDP", package, version, dry_run = True)
    

**conf_select.py**
    
    
    
    code, stdout = shell.call(get_cmd(command, package, version), logoutput=False, quiet=False,
sudo=True)
    

**Now the real question here is why in the world the `/etc/zookeeper/2.3.2.0-2950/0` directory
is being used a the cwd!** The workaround of this issue is to ensure we pass in the `cwd`
to `shell` which will pass it to `Popen`:
    
    
    
    code, stdout = shell.call(get_cmd(command, package, version), logoutput=False, quiet=False,
sudo=True, cwd="/")
    

I think we need to:

  * See if this can be reproduced and understood
  * Determine a proper value for `cwd` here


Diffs
-----

  ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py 6b7d09d

  ambari-server/src/test/python/stacks/2.0.6/HBASE/test_hbase_client.py fde28a5 
  ambari-server/src/test/python/stacks/2.0.6/HBASE/test_hbase_master.py 7602083 
  ambari-server/src/test/python/stacks/2.0.6/HBASE/test_hbase_regionserver.py ede200d 
  ambari-server/src/test/python/stacks/2.0.6/HBASE/test_phoenix_queryserver.py ddb111e 
  ambari-server/src/test/python/stacks/2.0.6/HDFS/test_datanode.py 350424c 
  ambari-server/src/test/python/stacks/2.0.6/HDFS/test_hdfs_client.py 0747eca 
  ambari-server/src/test/python/stacks/2.0.6/HDFS/test_journalnode.py 99b49a7 
  ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py ef3c5a5 
  ambari-server/src/test/python/stacks/2.0.6/HDFS/test_nfsgateway.py ecaa6c8 
  ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_client.py bca4034 
  ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_server.py 747a25d 
  ambari-server/src/test/python/stacks/2.0.6/HIVE/test_webhcat_server.py deb89e5 
  ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_client.py 9e3106f 
  ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_server.py 9c2efe6 
  ambari-server/src/test/python/stacks/2.0.6/PIG/test_pig_client.py dc24706 
  ambari-server/src/test/python/stacks/2.0.6/SQOOP/test_sqoop.py 765aba3 
  ambari-server/src/test/python/stacks/2.0.6/YARN/test_historyserver.py d361542 
  ambari-server/src/test/python/stacks/2.0.6/YARN/test_mapreduce2_client.py 95df52d 
  ambari-server/src/test/python/stacks/2.0.6/YARN/test_nodemanager.py 510070e 
  ambari-server/src/test/python/stacks/2.0.6/YARN/test_resourcemanager.py 26f6489 
  ambari-server/src/test/python/stacks/2.0.6/YARN/test_yarn_client.py 7615aed 
  ambari-server/src/test/python/stacks/2.0.6/ZOOKEEPER/test_zookeeper_client.py a77b25d 
  ambari-server/src/test/python/stacks/2.0.6/ZOOKEEPER/test_zookeeper_server.py 5cc5760 
  ambari-server/src/test/python/stacks/2.1/FALCON/test_falcon_client.py 1a7de71 
  ambari-server/src/test/python/stacks/2.1/FALCON/test_falcon_server.py ea8c859 
  ambari-server/src/test/python/stacks/2.1/HIVE/test_hive_metastore.py 7decf19 
  ambari-server/src/test/python/stacks/2.1/STORM/test_storm_drpc_server.py 1bedadd 
  ambari-server/src/test/python/stacks/2.1/STORM/test_storm_nimbus.py 13340ef 
  ambari-server/src/test/python/stacks/2.1/STORM/test_storm_nimbus_prod.py 4ee8d22 
  ambari-server/src/test/python/stacks/2.1/STORM/test_storm_supervisor.py 549571b 
  ambari-server/src/test/python/stacks/2.1/STORM/test_storm_supervisor_prod.py c21a81a 
  ambari-server/src/test/python/stacks/2.1/STORM/test_storm_ui_server.py abbbab4 
  ambari-server/src/test/python/stacks/2.1/TEZ/test_tez_client.py 59fedc4 
  ambari-server/src/test/python/stacks/2.1/YARN/test_apptimelineserver.py 2eca249 
  ambari-server/src/test/python/stacks/2.2/ACCUMULO/test_accumulo_client.py 37a5600 
  ambari-server/src/test/python/stacks/2.2/KAFKA/test_kafka_broker.py ea319f7 
  ambari-server/src/test/python/stacks/2.2/KNOX/test_knox_gateway.py 082aa2c 
  ambari-server/src/test/python/stacks/2.2/RANGER/test_ranger_admin.py 66a20a8 
  ambari-server/src/test/python/stacks/2.2/RANGER/test_ranger_usersync.py 4948ac7 
  ambari-server/src/test/python/stacks/2.2/SLIDER/test_slider_client.py 11fb429 
  ambari-server/src/test/python/stacks/2.2/SPARK/test_job_history_server.py bc34623 
  ambari-server/src/test/python/stacks/2.2/SPARK/test_spark_client.py e611acb 
  ambari-server/src/test/python/stacks/2.3/MAHOUT/test_mahout_client.py 867acc9 
  ambari-server/src/test/python/stacks/2.3/SPARK/test_spark_thrift_server.py f26ea31 

Diff: https://reviews.apache.org/r/40581/diff/


Testing
-------

mvn clean test


Thanks,

Andrew Onischuk


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message