ambari-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Siddharth Wagle (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (AMBARI-21624) HDFS restart failed post Ambari upgrade when lzo compression was enabled
Date Tue, 01 Aug 2017 18:50:00 GMT

     [ https://issues.apache.org/jira/browse/AMBARI-21624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Siddharth Wagle updated AMBARI-21624:
-------------------------------------
    Description: 
This is specifically for the IOP to HDP migration. We started with an IOP 4.2.5 cluster where
HDFS had "com.hadoop.compression.lzo" as part of the core-site/io.compression.codecs property
value.  After Ambari upgrade (build 2.5.2-155), we couldn't restart HDFS, where it failed
with the following error (bold/italic for the error line)

IOP 4.2.5 lzo package is hadoop-lzo-0.5.1-2.x86_64.rpm, which doesn't fit the naming convention
Ambari 2.5.2 uses (seems to be coming from HDP 2.3 stack via inheritance). It's also puzzling
why the inheritance as this was right after Ambari upgrade where the stack was still IOP 4.2.5
stack that extends common services directly.

{code}
2017-07-31 09:00:59,827 - Package['lzo'] {'retry_on_repo_unavailability': False, 'retry_count':
5}
2017-07-31 09:01:00,355 - Skipping installation of existing package lzo
2017-07-31 09:01:00,355 - Package['hadoop-lzo-native'] {'retry_on_repo_unavailability': False,
'retry_count': 5}
2017-07-31 09:01:00,381 - Skipping installation of existing package hadoop-lzo-native
2017-07-31 09:01:00,381 - Package['hadooplzo_4_2_5_IBM_1_1'] {'retry_on_repo_unavailability':
False, 'retry_count': 5}
*_2017-07-31 09:01:00,407 - Installing package hadooplzo_4_2_5_IBM_1_1 ('/usr/bin/yum -d 0
-e 0 -y install hadooplzo_4_2_5_IBM_1_1')
2017-07-31 09:01:01,212 - Execution of '/usr/bin/yum -d 0 -e 0 -y install hadooplzo_4_2_5_IBM_1_1'
returned 1. Error: Nothing to do_*
2017-07-31 09:01:01,212 - Failed to install package hadooplzo_4_2_5_IBM_1_1. Executing '/usr/bin/yum
clean metadata'
2017-07-31 09:01:01,553 - Retrying to install package hadooplzo_4_2_5_IBM_1_1 after 30 seconds

Command failed after 1 tries

core-site has io.compression.codecs = org.apache.hadoop.io.compress.GzipCodec,org.apache.hadoop.io.compress.DefaultCodec,org.apache.hadoop.io.compress.SnappyCodec,com.hadoop.compression.lzo.LzoCodec
{code}

  was:
This is specifically for the IOP to HDP migration. We started with an IOP 4.2.5 cluster where
HDFS had "com.hadoop.compression.lzo" as part of the core-site/io.compression.codecs property
value.  After Ambari upgrade (build 2.5.2-155), we couldn't restart HDFS, where it failed
with the following error (bold/italic for the error line)

IOP 4.2.5 lzo package is hadoop-lzo-0.5.1-2.x86_64.rpm, which doesn't fit the naming convention
Ambari 2.5.2 uses (seems to be coming from HDP 2.3 stack via inheritance). It's also puzzling
why the inheritance as this was right after Ambari upgrade where the stack was still IOP 4.2.5
stack that extends common services directly.

_2017-07-31 09:00:59,827 - Package['lzo'] {'retry_on_repo_unavailability': False, 'retry_count':
5}
2017-07-31 09:01:00,355 - Skipping installation of existing package lzo
2017-07-31 09:01:00,355 - Package['hadoop-lzo-native'] {'retry_on_repo_unavailability': False,
'retry_count': 5}
2017-07-31 09:01:00,381 - Skipping installation of existing package hadoop-lzo-native
2017-07-31 09:01:00,381 - Package['hadooplzo_4_2_5_IBM_1_1'] {'retry_on_repo_unavailability':
False, 'retry_count': 5}
*_2017-07-31 09:01:00,407 - Installing package hadooplzo_4_2_5_IBM_1_1 ('/usr/bin/yum -d 0
-e 0 -y install hadooplzo_4_2_5_IBM_1_1')
2017-07-31 09:01:01,212 - Execution of '/usr/bin/yum -d 0 -e 0 -y install hadooplzo_4_2_5_IBM_1_1'
returned 1. Error: Nothing to do_*
2017-07-31 09:01:01,212 - Failed to install package hadooplzo_4_2_5_IBM_1_1. Executing '/usr/bin/yum
clean metadata'
2017-07-31 09:01:01,553 - Retrying to install package hadooplzo_4_2_5_IBM_1_1 after 30 seconds

Command failed after 1 tries

core-site has io.compression.codecs = org.apache.hadoop.io.compress.GzipCodec,org.apache.hadoop.io.compress.DefaultCodec,org.apache.hadoop.io.compress.SnappyCodec,com.hadoop.compression.lzo.LzoCodec_


> HDFS restart failed post Ambari upgrade when lzo compression was enabled
> ------------------------------------------------------------------------
>
>                 Key: AMBARI-21624
>                 URL: https://issues.apache.org/jira/browse/AMBARI-21624
>             Project: Ambari
>          Issue Type: Bug
>          Components: ambari-server
>    Affects Versions: 2.5.2
>            Reporter: Di Li
>             Fix For: 2.5.2
>
>
> This is specifically for the IOP to HDP migration. We started with an IOP 4.2.5 cluster
where HDFS had "com.hadoop.compression.lzo" as part of the core-site/io.compression.codecs
property value.  After Ambari upgrade (build 2.5.2-155), we couldn't restart HDFS, where it
failed with the following error (bold/italic for the error line)
> IOP 4.2.5 lzo package is hadoop-lzo-0.5.1-2.x86_64.rpm, which doesn't fit the naming
convention Ambari 2.5.2 uses (seems to be coming from HDP 2.3 stack via inheritance). It's
also puzzling why the inheritance as this was right after Ambari upgrade where the stack was
still IOP 4.2.5 stack that extends common services directly.
> {code}
> 2017-07-31 09:00:59,827 - Package['lzo'] {'retry_on_repo_unavailability': False, 'retry_count':
5}
> 2017-07-31 09:01:00,355 - Skipping installation of existing package lzo
> 2017-07-31 09:01:00,355 - Package['hadoop-lzo-native'] {'retry_on_repo_unavailability':
False, 'retry_count': 5}
> 2017-07-31 09:01:00,381 - Skipping installation of existing package hadoop-lzo-native
> 2017-07-31 09:01:00,381 - Package['hadooplzo_4_2_5_IBM_1_1'] {'retry_on_repo_unavailability':
False, 'retry_count': 5}
> *_2017-07-31 09:01:00,407 - Installing package hadooplzo_4_2_5_IBM_1_1 ('/usr/bin/yum
-d 0 -e 0 -y install hadooplzo_4_2_5_IBM_1_1')
> 2017-07-31 09:01:01,212 - Execution of '/usr/bin/yum -d 0 -e 0 -y install hadooplzo_4_2_5_IBM_1_1'
returned 1. Error: Nothing to do_*
> 2017-07-31 09:01:01,212 - Failed to install package hadooplzo_4_2_5_IBM_1_1. Executing
'/usr/bin/yum clean metadata'
> 2017-07-31 09:01:01,553 - Retrying to install package hadooplzo_4_2_5_IBM_1_1 after 30
seconds
> Command failed after 1 tries
> core-site has io.compression.codecs = org.apache.hadoop.io.compress.GzipCodec,org.apache.hadoop.io.compress.DefaultCodec,org.apache.hadoop.io.compress.SnappyCodec,com.hadoop.compression.lzo.LzoCodec
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message