ambari-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeff Zhang <zjf...@gmail.com>
Subject Re: Is there any guide on how to write stackadvisor ?
Date Fri, 29 Jan 2016 05:39:39 GMT
Thanks all, these are all very helpful.

On Fri, Jan 29, 2016 at 10:15 AM, jun aoki <jaoki@apache.org> wrote:

> Hi Jeff,
>
> You may have already figured out, just another example here.
>
> https://github.com/Pivotal-Hadoop/ambari/blob/trunk/ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py#L67
> (I believe) This counts how many HAWQ segments ( HAWQ's slave component)
> there is in a cluster.
>
> On Tue, Jan 26, 2016 at 4:52 PM, Jeff Zhang <zjffdu@gmail.com> wrote:
>
>> Thanks Mithun and Srimanth. What I want to get the number of node
>> managers, the configured memory/cores of each node manager and the min/max
>> allocation memory of yarn, and base on these information I will recommend
>> configuration for spark-thrift-sparkconf.conf
>>
>> I will take a look the material you suggested.
>>
>> On Wed, Jan 27, 2016 at 2:50 AM, Srimanth Gunturi <
>> sgunturi@hortonworks.com> wrote:
>>
>>> Hi Jeff,
>>>
>>> The closest we have to documentation is the python doc comments we have
>>> in the base abstract class of StackAdvisor -
>>> https://github.com/apache/ambari/blob/trunk/ambari-server/src/main/resources/stacks/stack_advisor.py#L23
>>>
>>>
>>> Each stack advisor does 4 things for which there are 4 ​abstract methods
>>>
>>> 1. recommend component layout - *def** recommendComponentLayout(self,
>>> services, hosts)*
>>>
>>> 2. validate component layout - *def** validateComponentLayout(self,
>>> services, hosts)*
>>>
>>> 3. recommend configurations based on layout - *def**
>>> recommendConfigurations(self, services, hosts)*
>>>
>>> 4. validate configurations based on layout - *def**
>>> validateConfigurations(self, services, hosts)*
>>>
>>>
>>> If you notice, all 4 methods are passed 2 important pieces of
>>> information:
>>>
>>> 1. services - Contains any/all information about services being
>>> installed and which hosts they are installed on
>>>
>>> 2. hosts - Contains any/all information about hosts in cluster
>>>
>>>
>>> The above 2 data structures will tell you anything about the cluster
>>> being created/edited.
>>>
>>>
>>> So to the question about 'how to get the number of nodemangers' - it
>>> should be available by parsing both these data structures. We have utility
>>> method called 'def getHostsWithComponent(self, serviceName,
>>> componentName, services, hosts)' which gets the hosts a particular
>>> component is installed. You can see its usage at -
>>> https://github.com/apache/ambari/blob/trunk/ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py#L722
>>> ​
>>>
>>>
>>> Hoping this helps.
>>>
>>> Best regards,
>>>
>>> Srimanth
>>>
>>>
>>>
>>>
>>> ------------------------------
>>> *From:* Mithun Mathew <mithmatt@gmail.com>
>>> *Sent:* Monday, January 25, 2016 8:50 PM
>>> *To:* user@ambari.apache.org
>>> *Cc:* dev@ambari.apache.org
>>> *Subject:* Re: Is there any guide on how to write stackadvisor ?
>>>
>>> Hi Jeff
>>>
>>> Some existing implementation might help you give an idea of how
>>> stack_advisor is implemented to component layout recommendation:
>>> https://github.com/apache/ambari/blob/trunk/ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py#L25-#L51
>>>
>>> If you can explain what exactly you are trying to achieve, I might be
>>> able to help you with it.
>>>
>>> Regards
>>> Matt
>>>
>>> On Mon, Jan 25, 2016 at 4:31 PM, Jeff Zhang <zjffdu@gmail.com> wrote:
>>>
>>>> I try to add stackadvisor for one component. The logic is simple, but
>>>> don't know how to do that use ambari api. e.g. how to get the number of
>>>> nodemangers, how to get the number of hosts that with my service deployed.
>>>> If there's any python api doc, there would be very helpful.
>>>>
>>>>
>>>> --
>>>> Best Regards
>>>>
>>>> Jeff Zhang
>>>>
>>>
>>>
>>>
>>> --
>>> *Mithun Mathew* (Matt)
>>>
>>>    - www.linkedin.com/in/mithunmatt/
>>>
>>>
>>
>>
>> --
>> Best Regards
>>
>> Jeff Zhang
>>
>
>
>
> --
> -jun
>



-- 
Best Regards

Jeff Zhang

Mime
View raw message