brooklyn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thomas Bouron <thomas.bou...@cloudsoftcorp.com>
Subject Re: Sensors for ambari metrics
Date Thu, 17 Sep 2015 11:28:48 GMT
Hi Duncan.

+1, that seems like a good idea, I like it!

Best.

On Mon, 14 Sep 2015 at 10:19 Duncan Grant <duncan.grant@cloudsoftcorp.com>
wrote:

> Thomas,
>
> How about a mix of 2, and 4?
>
> We could automatically publish a map of sensors for each component.  This
> could used for feeding to monitoring or writing a new, ambari specific,
> policy in java.
> e.g. sensor ambari.metrics - type Map - value {hdfs: { usage: 10}, yarn:
> {load: 0.5}}
>
> We could also add a config property to AmbariCluster (called
> ambariMetricsSensors?) which would be a list of sensors to upgrade to full
> sensors.  The upgraded sensors would then be available to use with generic
> policies.
> e.g.
> ambariMetricsSensors:
> - ambari.metrics.hdfs.usage
>
> I think this would cover most use cases.
>
> Regards
>
> Duncan
>
>
> On Tue, 8 Sep 2015 at 17:59 Thomas Bouron <thomas.bouron@cloudsoftcorp.com
> >
> wrote:
>
> > +1.
> >
> > I think the best approach would be a mix of 1, 2 and 3, i.e. pulling
> > specific metrics such as memory load / usage which will be useful to spin
> > up nodes through policies. Then we could pull the full map of metrics per
> > component which will avoid to have 100+ sensors.
> >
> > I like the idea of 4 but I'm not sure if it's possible.
> >
> > Best.
> >
> > On Wed, 26 Aug 2015 at 11:55 Duncan Grant <
> duncan.grant@cloudsoftcorp.com>
> > wrote:
> >
> > > I would like to be able to add policies to a brooklyn-ambari blueprint
> > that
> > > can react to ambari metrics e.g. to add extra nodes when load reaches a
> > set
> > > level.  To do this I need sensors that read the metrics provided by
> > Ambari.
> > >
> > > Ambari provides a lot of metrics per deployed component and I'm not
> sure
> > > what the best brooklyn practice to make these available would be.  I
> > think
> > > the following are all possibilities:
> > > 1) On AmbariServer entity dynamically add a sensor for each individual
> > > metric but this would mean adding a couple of hundred sensors.
> > > 2) On AmbariServer entity dynamically add a sensor for each component's
> > > metrics with a value of type map.  I'm not sure if this is possible or
> > how
> > > easy it would then be to add a policy based on the value of items in
> the
> > > map.
> > > 3) Create a few specific sensors that will read specific values.  If
> > > someone needs more then they would need to extend java.
> > > 4) Create a generic Enricher that can be added in yaml and can be
> > > configured to dynamically add a sensor for a specific metric.  I think
> > this
> > > should be possible but would require some sort of json path value to
> > direct
> > > it to the correct metric.
> > >
> > > Does anyone have any advice on the best approach here?
> > >
> > > Regards
> > >
> > > Duncan
> > >
> > > --
> > > Cloudsoft Corporation Limited, Registered in Scotland No: SC349230.
> > >  Registered Office: 13 Dryden Place, Edinburgh, EH9 1RP
> > >
> > > This e-mail message is confidential and for use by the addressee only.
> If
> > > the message is received by anyone other than the addressee, please
> return
> > > the message to the sender by replying to it and then delete the message
> > > from your computer. Internet e-mails are not necessarily secure.
> > Cloudsoft
> > > Corporation Limited does not accept responsibility for changes made to
> > this
> > > message after it was sent.
> > >
> > > Whilst all reasonable care has been taken to avoid the transmission of
> > > viruses, it is the responsibility of the recipient to ensure that the
> > > onward transmission, opening or use of this message and any attachments
> > > will not adversely affect its systems or data. No responsibility is
> > > accepted by Cloudsoft Corporation Limited in this regard and the
> > recipient
> > > should carry out such virus and other checks as it considers
> appropriate.
> > >
> > --
> > Thomas Bouron • Software Engineer @ Cloudsoft Corporation •
> > http://www.cloudsoftcorp.com/
> > Github: https://github.com/tbouron
> > Twitter: https://twitter.com/eltibouron
> >
> > --
> > Cloudsoft Corporation Limited, Registered in Scotland No: SC349230.
> >  Registered Office: 13 Dryden Place, Edinburgh, EH9 1RP
> >
> > This e-mail message is confidential and for use by the addressee only. If
> > the message is received by anyone other than the addressee, please return
> > the message to the sender by replying to it and then delete the message
> > from your computer. Internet e-mails are not necessarily secure.
> Cloudsoft
> > Corporation Limited does not accept responsibility for changes made to
> this
> > message after it was sent.
> >
> > Whilst all reasonable care has been taken to avoid the transmission of
> > viruses, it is the responsibility of the recipient to ensure that the
> > onward transmission, opening or use of this message and any attachments
> > will not adversely affect its systems or data. No responsibility is
> > accepted by Cloudsoft Corporation Limited in this regard and the
> recipient
> > should carry out such virus and other checks as it considers appropriate.
> >
>
> --
> Cloudsoft Corporation Limited, Registered in Scotland No: SC349230.
>  Registered Office: 13 Dryden Place, Edinburgh, EH9 1RP
>
> This e-mail message is confidential and for use by the addressee only. If
> the message is received by anyone other than the addressee, please return
> the message to the sender by replying to it and then delete the message
> from your computer. Internet e-mails are not necessarily secure. Cloudsoft
> Corporation Limited does not accept responsibility for changes made to this
> message after it was sent.
>
> Whilst all reasonable care has been taken to avoid the transmission of
> viruses, it is the responsibility of the recipient to ensure that the
> onward transmission, opening or use of this message and any attachments
> will not adversely affect its systems or data. No responsibility is
> accepted by Cloudsoft Corporation Limited in this regard and the recipient
> should carry out such virus and other checks as it considers appropriate.
>
-- 
Thomas Bouron • Software Engineer @ Cloudsoft Corporation •
http://www.cloudsoftcorp.com/
Github: https://github.com/tbouron
Twitter: https://twitter.com/eltibouron

-- 
Cloudsoft Corporation Limited, Registered in Scotland No: SC349230. 
 Registered Office: 13 Dryden Place, Edinburgh, EH9 1RP
 
This e-mail message is confidential and for use by the addressee only. If 
the message is received by anyone other than the addressee, please return 
the message to the sender by replying to it and then delete the message 
from your computer. Internet e-mails are not necessarily secure. Cloudsoft 
Corporation Limited does not accept responsibility for changes made to this 
message after it was sent.

Whilst all reasonable care has been taken to avoid the transmission of 
viruses, it is the responsibility of the recipient to ensure that the 
onward transmission, opening or use of this message and any attachments 
will not adversely affect its systems or data. No responsibility is 
accepted by Cloudsoft Corporation Limited in this regard and the recipient 
should carry out such virus and other checks as it considers appropriate.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message