ambari-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeff Zhang <zjf...@gmail.com>
Subject Re: Is there any guide on how to write stackadvisor ?
Date Wed, 27 Jan 2016 00:52:55 GMT
Thanks Mithun and Srimanth. What I want to get the number of node managers,
the configured memory/cores of each node manager and the min/max allocation
memory of yarn, and base on these information I will recommend
configuration for spark-thrift-sparkconf.conf

I will take a look the material you suggested.

On Wed, Jan 27, 2016 at 2:50 AM, Srimanth Gunturi <sgunturi@hortonworks.com>
wrote:

> Hi Jeff,
>
> The closest we have to documentation is the python doc comments we have in
> the base abstract class of StackAdvisor -
> https://github.com/apache/ambari/blob/trunk/ambari-server/src/main/resources/stacks/stack_advisor.py#L23
>
>
> Each stack advisor does 4 things for which there are 4 ​abstract methods
>
> 1. recommend component layout - *def** recommendComponentLayout(self,
> services, hosts)*
>
> 2. validate component layout - *def** validateComponentLayout(self,
> services, hosts)*
>
> 3. recommend configurations based on layout - *def**
> recommendConfigurations(self, services, hosts)*
>
> 4. validate configurations based on layout - *def**
> validateConfigurations(self, services, hosts)*
>
>
> If you notice, all 4 methods are passed 2 important pieces of information:
>
> 1. services - Contains any/all information about services being installed
> and which hosts they are installed on
>
> 2. hosts - Contains any/all information about hosts in cluster
>
>
> The above 2 data structures will tell you anything about the cluster being
> created/edited.
>
>
> So to the question about 'how to get the number of nodemangers' - it
> should be available by parsing both these data structures. We have utility
> method called 'def getHostsWithComponent(self, serviceName, componentName,
> services, hosts)' which gets the hosts a particular component is
> installed. You can see its usage at -
> https://github.com/apache/ambari/blob/trunk/ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py#L722
> ​
>
>
> Hoping this helps.
>
> Best regards,
>
> Srimanth
>
>
>
>
> ------------------------------
> *From:* Mithun Mathew <mithmatt@gmail.com>
> *Sent:* Monday, January 25, 2016 8:50 PM
> *To:* user@ambari.apache.org
> *Cc:* dev@ambari.apache.org
> *Subject:* Re: Is there any guide on how to write stackadvisor ?
>
> Hi Jeff
>
> Some existing implementation might help you give an idea of how
> stack_advisor is implemented to component layout recommendation:
> https://github.com/apache/ambari/blob/trunk/ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py#L25-#L51
>
> If you can explain what exactly you are trying to achieve, I might be able
> to help you with it.
>
> Regards
> Matt
>
> On Mon, Jan 25, 2016 at 4:31 PM, Jeff Zhang <zjffdu@gmail.com> wrote:
>
>> I try to add stackadvisor for one component. The logic is simple, but
>> don't know how to do that use ambari api. e.g. how to get the number of
>> nodemangers, how to get the number of hosts that with my service deployed.
>> If there's any python api doc, there would be very helpful.
>>
>>
>> --
>> Best Regards
>>
>> Jeff Zhang
>>
>
>
>
> --
> *Mithun Mathew* (Matt)
>
>    - www.linkedin.com/in/mithunmatt/
>
>


-- 
Best Regards

Jeff Zhang

Mime
View raw message