ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Onischuk (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (AMBARI-14430) Get rid of wildcards in package names.
Date Fri, 18 Dec 2015 10:35:46 GMT

     [ https://issues.apache.org/jira/browse/AMBARI-14430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Andrew Onischuk updated AMBARI-14430:
-------------------------------------
    Description: 
**Problem**:  
Wildcards in packages bring and bunch of problems connected with RU and normal
installs.  
a) If we install let's say hadoop_2_2_* and hadoop_2_2_0_1234-libhdfs get
installed, but hadoop_2_2_0_1234 fails, we can no longer install master
package.  
b) Packages like hive_2_2-*-atlas-plugin, if for some reason they get
installed before hive_2_2_*, we get problems with inability to install. I've
bumped into a couple of different situations like this. (storm_2_2* and
storm_slider_2_2* etc. ; zookeeper-* and zookeeper-2_2*-server). That all 3
are real bugs which we had to hack out  
c) We install a lot of packages which are not needed by default like hadoop-
2_2_...fuse, just by asking hadoop_2_2_*.  
d) Situtations like 47931 can happen in RU.  
e) The bug in RU where we add a service after install_packages was triggered.

**Proposal:**  
1\. Replace * by ${hdp_version} in metainfo.xml.  
E.g.:

    
    
            <osSpecific>
              <osFamily>redhat7,redhat6,suse11</osFamily>
              <packages>
                <package>
                  <name>hbase_2_3_*</name>
                </package>
                <package>
                  <name>phoenix_2_3_*</name>
                </package>
              </packages>
            </osSpecific>
    

->
    
    
            <osSpecific>
              <osFamily>redhat7,redhat6,suse11</osFamily>
              <packages>
                <package>
                  <name>hbase_${hdp_version}</name>
                </package>
                <package>
                  <name>hbase_${hdp_version}-regionserver</name>
                </package>
                <package>
                  <name>phoenix_${hdp_version}</name>
                </package>
              </packages>
            </osSpecific>
    

Note: this step will also let us not override packages in metainfo.xml with
every HDP version to just change e.g. hbase_2_2_* to hbase_2_3_*.

2\. During INSTALL action (script.py) or during install packages
(install_packages.py) get the version we are currently using from hdp-select.

e.g. for centos:

    
    
    [root@andrew-1 ~]# yum info hdp-select | grep 'Version' | awk '{print $3}'
    2.3.4.0
    [root@andrew-1 ~]# yum info hdp-select | grep 'Release' | awk '{print $3}'
    3204 # a bit slow for every INSTALL task
    

Another better aproach is to install hdp-select and than use rpm/dpkg, which
is faster than yum/zypper/apt-get, considering we will do this on every
INSTALL task this is important.

    
    
    [root@andrew-1 ~]# rpm -q --queryformat '%{version}-%{release}' hdp-select | sed -e 's/\.el[0-9]//g'
# faster approach
    2.3.4.0-3204
    
    
    
    u1201:~ # dpkg -s hdp-select | grep 'Version' | awk '{print $2}' 
    2.3.4.0-3475
    

Vs other approaches:  
I think it's the only possible approach to get the version here, sending
version which is specified by user during creating of a version. Seems to be a
bad idea as the user can specify wrong version. Also we will need to use this
during fresh install, where we don't have it specified at all.

3\. Replace ${hdp_version} with the version we got in step 2.

I also discussed this with dlysnichenko this seems to be feasible in terms of
RU/EU.  
The overall idea was discussed in 47931  
cc smohanty

Also after talking to dlysnichenko he said we were excluding repos because of issues with
wildcard on suse when it is not excluded and mutptiple packages is found for a wildcard. That's
why when we stop using wildcards we can remove those exclude repos, which as well caused a
bunch of bugs. 

  was:
**Problem**:  
Wildcards in packages bring and bunch of problems connected with RU and normal
installs.  
a) If we install let's say hadoop_2_2_* and hadoop_2_2_0_1234-libhdfs get
installed, but hadoop_2_2_0_1234 fails, we can no longer install master
package.  
b) Packages like hive_2_2-*-atlas-plugin, if for some reason they get
installed before hive_2_2_*, we get problems with inability to install. I've
bumped into a couple of different situations like this. (storm_2_2* and
storm_slider_2_2* etc. ; zookeeper-* and zookeeper-2_2*-server). That all 3
are real bugs which we had to hack out  
c) We install a lot of packages which are not needed by default like hadoop-
2_2_...fuse, just by asking hadoop_2_2_*.  
d) Situtations like 47931 can happen in RU.  
e) The bug in RU where we add a service after install_packages was triggered.

**Proposal:**  
1\. Replace * by ${hdp_version} in metainfo.xml.  
E.g.:

    
    
            <osSpecific>
              <osFamily>redhat7,redhat6,suse11</osFamily>
              <packages>
                <package>
                  <name>hbase_2_3_*</name>
                </package>
                <package>
                  <name>phoenix_2_3_*</name>
                </package>
              </packages>
            </osSpecific>
    

->
    
    
            <osSpecific>
              <osFamily>redhat7,redhat6,suse11</osFamily>
              <packages>
                <package>
                  <name>hbase_${hdp_version}</name>
                </package>
                <package>
                  <name>hbase_${hdp_version}-regionserver</name>
                </package>
                <package>
                  <name>phoenix_${hdp_version}</name>
                </package>
              </packages>
            </osSpecific>
    

Note: this step will also let us not override packages in metainfo.xml with
every HDP version to just change e.g. hbase_2_2_* to hbase_2_3_*.

2\. During INSTALL action (script.py) or during install packages
(install_packages.py) get the version we are currently using from hdp-select.

e.g. for centos:

    
    
    [root@andrew-1 ~]# yum info hdp-select | grep 'Version' | awk '{print $3}'
    2.3.4.0
    [root@andrew-1 ~]# yum info hdp-select | grep 'Release' | awk '{print $3}'
    3204 # a bit slow for every INSTALL task
    

Another better aproach is to install hdp-select and than use rpm/dpkg, which
is faster than yum/zypper/apt-get, considering we will do this on every
INSTALL task this is important.

    
    
    [root@andrew-1 ~]# rpm -q --queryformat '%{version}-%{release}' hdp-select | sed -e 's/\.el[0-9]//g'
# faster approach
    2.3.4.0-3204
    
    
    
    u1201:~ # dpkg -s hdp-select | grep 'Version' | awk '{print $2}' 
    2.3.4.0-3475
    

Vs other approaches:  
I think it's the only possible approach to get the version here, sending
version which is specified by user during creating of a version. Seems to be a
bad idea as the user can specify wrong version. Also we will need to use this
during fresh install, where we don't have it specified at all.

3\. Replace ${hdp_version} with the version we got in step 2.

I also discussed this with dlysnichenko this seems to be feasible in terms of
RU/EU.  
The overall idea was discussed in 47931  
cc smohanty




> Get rid of wildcards in package names.
> --------------------------------------
>
>                 Key: AMBARI-14430
>                 URL: https://issues.apache.org/jira/browse/AMBARI-14430
>             Project: Ambari
>          Issue Type: Bug
>            Reporter: Andrew Onischuk
>            Assignee: Andrew Onischuk
>             Fix For: 2.4.0
>
>         Attachments: AMBARI-14430.patch
>
>
> **Problem**:  
> Wildcards in packages bring and bunch of problems connected with RU and normal
> installs.  
> a) If we install let's say hadoop_2_2_* and hadoop_2_2_0_1234-libhdfs get
> installed, but hadoop_2_2_0_1234 fails, we can no longer install master
> package.  
> b) Packages like hive_2_2-*-atlas-plugin, if for some reason they get
> installed before hive_2_2_*, we get problems with inability to install. I've
> bumped into a couple of different situations like this. (storm_2_2* and
> storm_slider_2_2* etc. ; zookeeper-* and zookeeper-2_2*-server). That all 3
> are real bugs which we had to hack out  
> c) We install a lot of packages which are not needed by default like hadoop-
> 2_2_...fuse, just by asking hadoop_2_2_*.  
> d) Situtations like 47931 can happen in RU.  
> e) The bug in RU where we add a service after install_packages was triggered.
> **Proposal:**  
> 1\. Replace * by ${hdp_version} in metainfo.xml.  
> E.g.:
>     
>     
>             <osSpecific>
>               <osFamily>redhat7,redhat6,suse11</osFamily>
>               <packages>
>                 <package>
>                   <name>hbase_2_3_*</name>
>                 </package>
>                 <package>
>                   <name>phoenix_2_3_*</name>
>                 </package>
>               </packages>
>             </osSpecific>
>     
> ->
>     
>     
>             <osSpecific>
>               <osFamily>redhat7,redhat6,suse11</osFamily>
>               <packages>
>                 <package>
>                   <name>hbase_${hdp_version}</name>
>                 </package>
>                 <package>
>                   <name>hbase_${hdp_version}-regionserver</name>
>                 </package>
>                 <package>
>                   <name>phoenix_${hdp_version}</name>
>                 </package>
>               </packages>
>             </osSpecific>
>     
> Note: this step will also let us not override packages in metainfo.xml with
> every HDP version to just change e.g. hbase_2_2_* to hbase_2_3_*.
> 2\. During INSTALL action (script.py) or during install packages
> (install_packages.py) get the version we are currently using from hdp-select.
> e.g. for centos:
>     
>     
>     [root@andrew-1 ~]# yum info hdp-select | grep 'Version' | awk '{print $3}'
>     2.3.4.0
>     [root@andrew-1 ~]# yum info hdp-select | grep 'Release' | awk '{print $3}'
>     3204 # a bit slow for every INSTALL task
>     
> Another better aproach is to install hdp-select and than use rpm/dpkg, which
> is faster than yum/zypper/apt-get, considering we will do this on every
> INSTALL task this is important.
>     
>     
>     [root@andrew-1 ~]# rpm -q --queryformat '%{version}-%{release}' hdp-select | sed
-e 's/\.el[0-9]//g' # faster approach
>     2.3.4.0-3204
>     
>     
>     
>     u1201:~ # dpkg -s hdp-select | grep 'Version' | awk '{print $2}' 
>     2.3.4.0-3475
>     
> Vs other approaches:  
> I think it's the only possible approach to get the version here, sending
> version which is specified by user during creating of a version. Seems to be a
> bad idea as the user can specify wrong version. Also we will need to use this
> during fresh install, where we don't have it specified at all.
> 3\. Replace ${hdp_version} with the version we got in step 2.
> I also discussed this with dlysnichenko this seems to be feasible in terms of
> RU/EU.  
> The overall idea was discussed in 47931  
> cc smohanty
> Also after talking to dlysnichenko he said we were excluding repos because of issues
with wildcard on suse when it is not excluded and mutptiple packages is found for a wildcard.
That's why when we stop using wildcards we can remove those exclude repos, which as well caused
a bunch of bugs. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message