ambari-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aravindan Vijayan <avija...@hortonworks.com>
Subject Re: Ambari metrics monitor-collector over the WAN
Date Thu, 05 Oct 2017 22:01:39 GMT
Hello,

It is one way SSL. The metric monitors (clients) verifies the metrics collector’s certificate
and then ‘POST’s the metrics.

--
Thanks and Regards,
Aravindan Vijayan

From: Ali Nazemian <alinazemian@gmail.com>
Reply-To: "user@ambari.apache.org" <user@ambari.apache.org>
Date: Thursday, October 5, 2017 at 2:48 PM
To: "user@ambari.apache.org" <user@ambari.apache.org>
Cc: "qinliu5678@gmail.com" <qinliu5678@gmail.com>
Subject: Re: Ambari metrics monitor-collector over the WAN

Perfect. Thank you very much. Regarding https, is it 2-way or 1 way?

On 6 Oct. 2017 04:30, "Siddharth Wagle" <swagle@hortonworks.com<mailto:swagle@hortonworks.com>>
wrote:

1- How does it scale for thousands of agents?



With Ambari 2.5.2 you can launch a 2 node collector in distributed mode and select the placement
of the node based on geographical location of the agents.

Unfortunately, we shard based on hostname to select from the monitor, which collector to write
to.

Presently this is not customizable however, if you need it to be feel free to open a Jira,
it is would be a straightforward fix required. However, I doubt you would need to do this
because all sinks locally aggregate metrics and write only after 1 minute of data is collected.



https://issues.apache.org/jira/browse/AMBARI-15901



Regarding volume of writes: We have tested the system with 2 node collector upto 2K nodes
and we already know of production systems of up to 1500 node working with AMS.

Since the collection model is push based the bottleneck needs to be solved on the collector
and therefore in 2.5.1+ version of Ambari you can add additional collector node to handle
extra load.



Note: In 3.0 we are doing quite a few things to address performance by de-centralizing aggregations
and reducing memory bloat on RegionServers etc, but that will be ready later in the year.



2- Can I use it over WAN probably by securing that in a two-way SSL way?



Regarding security: We already support https for AMS - https://issues.apache.org/jira/browse/AMBARI-15050



The SPNEGO based authentication work is still not complete: We have a quite a few patches
already in trunk but haven't looked at release readiness for these.

Possible Qin Liu can shed more light on this.



BR,

Sid



________________________________
From: Ali Nazemian <alinazemian@gmail.com<mailto:alinazemian@gmail.com>>
Sent: Thursday, October 5, 2017 1:53 AM
To: user@ambari.apache.org<mailto:user@ambari.apache.org>
Subject: Ambari metrics monitor-collector over the WAN

Hi all,

I am investigating to have a huge cluster of Nifi instances distributed across the world.
I have been told that Metrics Monitor and Metrics Collector are using a very chatty protocol,
so Ambari Metrics Collector may face some serious performance issues regarding receiving telemetry
information from lots of Metrics Monitor Agents. I have two concerns regarding that as follows.


1- How does it scale for thousands of agents?

2- Can I use it over WAN probably by securing that in a two-way SSL way?



Regards,

Ali
Mime
View raw message