ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Juanjo Marron <juanjo.mar...@yahoo.com.INVALID>
Subject Re: Managing stack_version on Ambari 2.4 code
Date Wed, 27 Jul 2016 01:33:40 GMT
Hi Alejandro, I think we found the solution to the issue, 
 
 To get the right stack_version value, script.py code checks for:

1) installed path /usr/<distro>/<version>. Our version was in the format x.x.x.x
and the regex in stack_select.py:get_stack_version_before_install expects x.x.x.x-yyy

2) <distro>-select rpm version. Our iop-select rpm has not exactly the same version
of the stack as  is your case

Couple of things that need to be tuned on pour side to follow the new requirements in Ambari-2.4

Hacking  and creating a symlink /usr/iop/4.3.0.0-1234 -> /usr/iop/4.3.0.0 solve the initial
issues or components  reprting their versions and current cluster state

Thanks for sharing your brain
  

    On Tuesday, July 26, 2016 5:17 PM, Alejandro Fernandez <afernandez@hortonworks.com>
wrote:
 

 Hi Juanjo, is that the only host in the system?
If you restart ZK, it should advertise the version back and update the
record in the hostcomponentstate table.
You can also check the structured-out-*.json file inside
/var/lib/ambari-agent/data that corresponds to the restart command, and it
should contain something like,

{"version": "2.3.0.0-1234", "repository_version_id": "1"}

Make sure that ZK's metainfo.xml has
<versionAdvertised>true</versionAdvertised>

Thanks,
Alejandro

On 7/26/16, 5:02 PM, "Juanjo Marron" <juanjo.marron@yahoo.com.INVALID>
wrote:

> Hi Alejandro,
> 
> Thanks a lot for your reply,
> 
>  I run a basic test just with Zookeeper in one node cluster. It is
>installed properly but when trying to add new services,  the restAPI call
>to obtain stack_services filtering by state=CURRENT is of course failing
>(state is INIT)
>
>  Due to this issue I started my investigation based on versions issues
>and I got the details in previous email:
>  - Components installed with unknown version
>  - missing current_version in command-xx.json
>  - missing stack_version in structured-out-XX.json
>
>The output of the DB queries are:
>
> Just one repo version with version 4.3
> 
> ambari=> SELECT repo_version_id, version FROM repo_version;
> repo_version_id | version
>-----------------+---------
>              1 | 4.3
>              
>              
> The right repo_version is used, the cluster never transitioned to CURRENT
> 
> ambari=> SELECT * FROM cluster_version;
> id | repo_version_id | cluster_id | state |  start_time  |  end_time
> | user_name
>----+-----------------+------------+-------+---------------+--------------
>-+-----------
>  1 |              1 |          2 | INIT  | 1469549875153 |
>1469549875153 | admin
>(1 row)
>
>Just a ZK components installed and versions are unknown
>
>ambari=> SELECT h.host_name, hcs.service_name, hcs.component_name,
>hcs.version FROM
>ambari-> hostcomponentstate hcs JOIN hosts h ON hcs.host_id = h.host_id
>ORDER BY
>ambari-> hcs.version, hcs.service_name, hcs.component_name, h.host_name;
>      host_name      | service_name |  component_name  | version
>---------------------+--------------+------------------+---------
> spins1.fyre.ibm.com | ZOOKEEPER    | ZOOKEEPER_CLIENT | UNKNOWN
> spins1.fyre.ibm.com | ZOOKEEPER    | ZOOKEEPER_SERVER | UNKNOWN
>(2 rows)
>              
>
> iop-select, as implementation of the new stack-selector, logic seems to
>report the right version:
> 
>[root@spins1 data]# /usr/bin/iop-select status zookeeper-client
>zookeeper-client - 4.3.0.0
>[root@spins1 data]# /usr/bin/iop-select status zookeeper-server
>zookeeper-server - 4.3.0.0
>
> So I guess the question now is, why the components are not advertising
>the correct version and they are marked as version = unknown even after
>installation?
>
> I saw versions for HDP rpms and installed components are  in the format
>2.5.0.0-965. 
> 
> peggs1.fyre.ibm.com | ZOOKEEPER      | ZOOKEEPER_SERVER    | 2.5.0.0-965
> peggs2.fyre.ibm.com | ZOOKEEPER      | ZOOKEEPER_SERVER    | 2.5.0.0-965
> peggs3.fyre.ibm.com | ZOOKEEPER      | ZOOKEEPER_SERVER    | 2.5.0.0-965
>
> Is this expected in the rpm name/version?  Are we missing some
>x.x.x.x-yyy required format in Ambari 2.4 and the parse could be failing?
> 
> I also realized that now the packages to be installed (as defined in
>service metainfo.xml) require the stack version (it used to be wildcards
>* on Ambari-2.2.2)
> 
>            <package>
>              <name>zookeeper_${stack_version}</name>
>            </package>
>
>How are they connected?
>            
>Once again thanks for your insights Alejandro
> 
>
>    On Tuesday, July 26, 2016 4:05 PM, Alejandro Fernandez
><afernandez@hortonworks.com> wrote:
> 
>
> Hi Juanjo, in order for the cluster_version to transition to CURRENT as
>part of the installation, all of the components in the stack that
>advertise a version (through some means like hdp-select) must actually
>start up on that version.
>Can you provide more details?
>
>SELECT repo_version_id, version FROM repo_version;
>SELECT * FROM cluster_version;
>SELECT h.host_name, hcs.service_name, hcs.component_name, hcs.version FROM
>hostcomponentstate hcs JOIN hosts h ON hcs.host_id = h.host_id ORDER BY
>hcs.version, hcs.service_name, hcs.component_name, h.host_name;
>
>Thanks,
>Alejandro
>
>On 7/26/16, 2:37 PM, "Juanjo Marron" <juanjo.marron@yahoo.com.INVALID>
>wrote:
>
>>Hi all,
>>
>>Last days I was working on creating a new stack definition based on
>>branch-2.4 code and targeting to test new stack featurization and
>>extensibility features.
>>
>>The main goal is to create a new stack by reusing current common-services
>>code and changing installation path (different to /usr/hdp/) and stack
>>version number (diff to 2.5)
>>
>>Once the stack was created, I was installing same service rpms we used in
>>previous distribution based on Ambari-2.2 and  im hitting  issues with
>>cluster state set to current on DB (state is still INIT). Actually the
>>components installed maintain their version as unknown in DB and hosts
>>state is also out of sync what seems due to same version issue.
>>Additionally current_version is never included in hostLevel params in
>>command-xx.json file and structured-out-XX.json doesnt show the
>>stack_version either
>>
>>Is there any important change in how rpms for Ambari-2.4 need to be build
>>in terms of versioning compared with Ambari-2.2.2 and previous versions?
>>
>>Are there any spec files to build rpms available as reference in some
>>other repo or as part of the Ambari code?
>> It seems they are strongly coupled to what Ambari framework and stacks
>>are expecting. Some documentation where I can read about it.
>>
>>I can provide more details about the current test if required, but I
>>think at this point I am missing some piece in the management of the
>>stack version between rpms and Ambari framework
>>
>>I would really appreciate any help or comment
>>
>>Thanks all for your time
>>
>
>
>  


  
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message