ambari-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jun aoki <ja...@apache.org>
Subject Re: Is there any guide on how to write stackadvisor ?
Date Fri, 29 Jan 2016 02:15:41 GMT
Hi Jeff,

You may have already figured out, just another example here.
https://github.com/Pivotal-Hadoop/ambari/blob/trunk/ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py#L67
(I believe) This counts how many HAWQ segments ( HAWQ's slave component)
there is in a cluster.

On Tue, Jan 26, 2016 at 4:52 PM, Jeff Zhang <zjffdu@gmail.com> wrote:

> Thanks Mithun and Srimanth. What I want to get the number of node
> managers, the configured memory/cores of each node manager and the min/max
> allocation memory of yarn, and base on these information I will recommend
> configuration for spark-thrift-sparkconf.conf
>
> I will take a look the material you suggested.
>
> On Wed, Jan 27, 2016 at 2:50 AM, Srimanth Gunturi <
> sgunturi@hortonworks.com> wrote:
>
>> Hi Jeff,
>>
>> The closest we have to documentation is the python doc comments we have
>> in the base abstract class of StackAdvisor -
>> https://github.com/apache/ambari/blob/trunk/ambari-server/src/main/resources/stacks/stack_advisor.py#L23
>>
>>
>> Each stack advisor does 4 things for which there are 4 ​abstract methods
>>
>> 1. recommend component layout - *def** recommendComponentLayout(self,
>> services, hosts)*
>>
>> 2. validate component layout - *def** validateComponentLayout(self,
>> services, hosts)*
>>
>> 3. recommend configurations based on layout - *def**
>> recommendConfigurations(self, services, hosts)*
>>
>> 4. validate configurations based on layout - *def**
>> validateConfigurations(self, services, hosts)*
>>
>>
>> If you notice, all 4 methods are passed 2 important pieces of information:
>>
>> 1. services - Contains any/all information about services being installed
>> and which hosts they are installed on
>>
>> 2. hosts - Contains any/all information about hosts in cluster
>>
>>
>> The above 2 data structures will tell you anything about the cluster
>> being created/edited.
>>
>>
>> So to the question about 'how to get the number of nodemangers' - it
>> should be available by parsing both these data structures. We have utility
>> method called 'def getHostsWithComponent(self, serviceName, componentName,
>> services, hosts)' which gets the hosts a particular component is
>> installed. You can see its usage at -
>> https://github.com/apache/ambari/blob/trunk/ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py#L722
>> ​
>>
>>
>> Hoping this helps.
>>
>> Best regards,
>>
>> Srimanth
>>
>>
>>
>>
>> ------------------------------
>> *From:* Mithun Mathew <mithmatt@gmail.com>
>> *Sent:* Monday, January 25, 2016 8:50 PM
>> *To:* user@ambari.apache.org
>> *Cc:* dev@ambari.apache.org
>> *Subject:* Re: Is there any guide on how to write stackadvisor ?
>>
>> Hi Jeff
>>
>> Some existing implementation might help you give an idea of how
>> stack_advisor is implemented to component layout recommendation:
>> https://github.com/apache/ambari/blob/trunk/ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py#L25-#L51
>>
>> If you can explain what exactly you are trying to achieve, I might be
>> able to help you with it.
>>
>> Regards
>> Matt
>>
>> On Mon, Jan 25, 2016 at 4:31 PM, Jeff Zhang <zjffdu@gmail.com> wrote:
>>
>>> I try to add stackadvisor for one component. The logic is simple, but
>>> don't know how to do that use ambari api. e.g. how to get the number of
>>> nodemangers, how to get the number of hosts that with my service deployed.
>>> If there's any python api doc, there would be very helpful.
>>>
>>>
>>> --
>>> Best Regards
>>>
>>> Jeff Zhang
>>>
>>
>>
>>
>> --
>> *Mithun Mathew* (Matt)
>>
>>    - www.linkedin.com/in/mithunmatt/
>>
>>
>
>
> --
> Best Regards
>
> Jeff Zhang
>



-- 
-jun

Mime
View raw message