Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 9C186200B13 for ; Wed, 15 Jun 2016 22:03:58 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 9A825160A4D; Wed, 15 Jun 2016 20:03:58 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 9804D160A19 for ; Wed, 15 Jun 2016 22:03:57 +0200 (CEST) Received: (qmail 70471 invoked by uid 500); 15 Jun 2016 20:03:56 -0000 Mailing-List: contact user-help@ambari.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@ambari.apache.org Delivered-To: mailing list user@ambari.apache.org Received: (qmail 70461 invoked by uid 99); 15 Jun 2016 20:03:56 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 15 Jun 2016 20:03:56 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 4993A18050F for ; Wed, 15 Jun 2016 20:03:56 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.798 X-Spam-Level: ** X-Spam-Status: No, score=2.798 tagged_above=-999 required=6.31 tests=[FSL_HELO_BARE_IP_2=1.499, HTML_MESSAGE=2, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id KM-NcM7-e1f7 for ; Wed, 15 Jun 2016 20:03:54 +0000 (UTC) Received: from relayvx11b.securemail.intermedia.net (relayvx11b.securemail.intermedia.net [64.78.52.184]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id 3EA285F297 for ; Wed, 15 Jun 2016 20:03:53 +0000 (UTC) Received: from securemail.intermedia.net (localhost [127.0.0.1]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by emg-ca-1-1.localdomain (Postfix) with ESMTPS id A59BF53E88; Wed, 15 Jun 2016 13:03:51 -0700 (PDT) Subject: Re: HDP install issues about hdp-select MIME-Version: 1.0 x-echoworx-msg-id: 58c2165f-bfcc-46c6-85a5-0b616feaaa79 x-echoworx-emg-received: Wed, 15 Jun 2016 13:03:51.621 -0700 x-echoworx-message-code-hashed: a928838bd0d479dd06efcf2511c6e5dd4ffbfc884429384d0d86fa6dc3444189 x-echoworx-action: delivered Received: from 10.254.155.14 ([10.254.155.14]) by emg-ca-1-1 (JAMES SMTP Server 2.3.2) with SMTP ID 93; Wed, 15 Jun 2016 13:03:51 -0700 (PDT) Received: from MBX080-W4-CO-2.exch080.serverpod.net (unknown [10.224.117.102]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by emg-ca-1-1.localdomain (Postfix) with ESMTPS id 6132C53F32; Wed, 15 Jun 2016 13:03:51 -0700 (PDT) Received: from MBX080-W4-CO-2.exch080.serverpod.net (10.224.117.102) by MBX080-W4-CO-2.exch080.serverpod.net (10.224.117.102) with Microsoft SMTP Server (TLS) id 15.0.1178.4; Wed, 15 Jun 2016 13:03:50 -0700 Received: from MBX080-W4-CO-2.exch080.serverpod.net ([10.224.117.102]) by mbx080-w4-co-2.exch080.serverpod.net ([10.224.117.102]) with mapi id 15.00.1178.000; Wed, 15 Jun 2016 13:03:50 -0700 From: Jonathan Hurley To: "user@ambari.apache.org" CC: Derek Palma Thread-Topic: HDP install issues about hdp-select Thread-Index: AQHRxxzLjVcctjC+GEGT9z4LF9W0Ep/q53L8gACBvIA= Date: Wed, 15 Jun 2016 20:03:49 +0000 Message-ID: <0425CE0C-BA4B-4538-86CF-6A0380D0113D@hortonworks.com> References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-ms-exchange-transport-fromentityheader: Hosted x-originating-ip: [173.61.90.63] x-source-routing-agent: Processed Content-Type: multipart/alternative; boundary="_000_0425CE0CBA4B453886CF6A0380D0113Dhortonworkscom_" archived-at: Wed, 15 Jun 2016 20:03:58 -0000 --_000_0425CE0CBA4B453886CF6A0380D0113Dhortonworkscom_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable I believe this is because you have a "hadoop" directory in /usr/hdp ... /us= r/hdp should only contain versions and "current". If there's another direct= ory, it would cause the hdp-select tool to fail. On Jun 15, 2016, at 3:23 PM, Pawel Akonom > wrote: Hi, I have some problems with hadoop cluster scratch installation (not upgrade)= . Versions I am using: Ambari version: 2.2.2.0 HDP stack version: 2.3.4.7 Zookeper installation fails on step: Execute['ambari-sudo.sh /usr/bin/hdp-select set all `ambari-python-wrap /us= r/bin/hdp-select versions | grep ^2.3 | tail -1`'] {'only_if': 'ls -d /usr/= hdp/2.3*'} When I execute this command manually on bash I get error: [root@hdp-vora-master ~]# ambari-python-wrap /usr/bin/hdp-select versions Traceback (most recent call last): File "/usr/bin/hdp-select", line 378, in printVersions() File "/usr/bin/hdp-select", line 235, in printVersions result[tuple(map(int, versionRegex.split(f)))] =3D f ValueError: invalid literal for int() with base 10: 'hadoop' The same problem is described in Hortonworks community: https://community.hortonworks.com/questions/5811/install-of-hdp-fails-with-= valueerror-invalid-liter.html Workaround for this problem is to edit /usr/bin/hdp-select python script an= d modify printVersions() function. Function before: # Print the installed packages def printVersions(): result =3D {} for f in os.listdir(root): if f not in [".", "..", "current", "share", "lost+found"]: result[tuple(map(int, versionRegex.split(f)))] =3D f keys =3D result.keys() keys.sort() for k in keys: print result[k] Function after modification: # Print the installed packages def printVersions(): result =3D {} for f in os.listdir(root): if f not in [".", "..", "current", "share", "lost+found"]: try: result[tuple(map(int, versionRegex.split(f)))] =3D f except: pass keys =3D result.keys() keys.sort() for k in keys: print result[k] Hadoop cluster installation need to be automated so this workaround is not = a solution. Script /usr/bin/hdp-select appears during installation and it d= oesn't come with any rpm package. It can be a bug in the script or maybe it= fails only with some specific python versions only. Do you know the problem? If so what can be the solution? Thanks in advance, Pawel --_000_0425CE0CBA4B453886CF6A0380D0113Dhortonworkscom_ Content-Type: text/html; charset="iso-8859-1" Content-ID: <311F93CB4638C046BF586E9019DC65B2@exch080.serverpod.net> Content-Transfer-Encoding: quoted-printable
I believe this is because you have a "hadoop" dir= ectory in /usr/hdp ... /usr/hdp should only contain versions and "curr= ent". If there's another directory, it would cause the hdp-select tool= to fail. 

On Jun 15, 2016, at 3:23 PM, Pawel Akonom <pawel.akonom@vnomic.com> wr= ote:

Hi, 

I have some = problems with hadoop cluster scratch installation (not upgrade). Versions I= am using:

Ambari version: 2.2.2.0
HDP stack= version: 2.3.4.7


Zookeper ins= tallation fails on step:

Execute['ambari-sudo= .sh /usr/bin/hdp-select set all `ambari-python-wrap /usr/bin/hdp-select ver= sions | grep ^2.3 | tail -1`'] {'only_if': 'ls -d /usr/hdp/2.3*'}

When I execu= te this command manually on bash I get error:

[root@hdp-vora-master ~]# ambari-python-wrap /usr/bin/hdp-select ver= sions
Traceback= (most recent call last):
  Fi= le "/usr/bin/hdp-select", line 378, in <module>
 &nb= sp;  printVersions()
  Fi= le "/usr/bin/hdp-select", line 235, in printVersions
 &nb= sp;  result[tuple(map(int, versionRegex.split(f)))] =3D f
ValueErro= r: invalid literal for int() with base 10: 'hadoop'

The same problem is described in Hortonworks community:
https://communit= y.hortonworks.com/questions/5811/install-of-hdp-fails-with-valueerror-inval= id-liter.html

Workaround for this problem is to edit /usr/bin/hdp-select python script and modify printVersions()&nbs= p;function.

Function before:

# Print the installed packages
def print= Versions():
  re= sult =3D {}
  fo= r f in os.listdir(root):
 &nb= sp;  if f not in [".", "..", "current", = "share", "lost+found"]:
 &nb= sp;    result[tuple(map(int, versionRegex.split(f)))] =3D f
  ke= ys =3D result.keys()
  ke= ys.sort()
  fo= r k in keys:
 &nb= sp;   print result[k]

Function after modification:

# Print the installed packages
def print= Versions():
  re= sult =3D {}
  fo= r f in os.listdir(root):
 &nb= sp;  if f not in [".", "..", "current", = "share", "lost+found"]:
      try:
        result[tuple(map(i= nt, versionRegex.split(f)))] =3D f
      except:
        pass
  ke= ys =3D result.keys()
  ke= ys.sort()
  fo= r k in keys:
 &nb= sp;   print result[k]

Hadoop cluster installation need to be automated so this workaround is not = a solution. Script /usr/bin/hdp-select appears during installation and it doesn't come with any rpm package. It can be a = bug in the script or maybe it fails only with some specific python versions= only.

Do you know the problem? If so what can be the solution?
Thanks in advance,
Pawel

--_000_0425CE0CBA4B453886CF6A0380D0113Dhortonworkscom_--