metron-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Juan Zamora (Jira)" <j...@apache.org>
Subject [jira] [Created] (METRON-2359) Metron MAAS Service High Availability
Date Tue, 02 Jun 2020 22:04:00 GMT
Juan Zamora created METRON-2359:
-----------------------------------

             Summary: Metron MAAS Service High Availability
                 Key: METRON-2359
                 URL: https://issues.apache.org/jira/browse/METRON-2359
             Project: Metron
          Issue Type: Bug
            Reporter: Juan Zamora
             Fix For: 0.7.0


Greetings

I am building a Stellar MAAS machine learning service to process some data. I am concerned
about the availability of the deployed service.

The MAAS deploy .sh allows me to define memory allocated as well as the running containers.
However, after this, it seems that YARN and Zookeeper takes care of the rest... 

Particularly, if one instance of the service is taking a lot of the data from the pipeline,
how does YARN or Zookeeper transfers the load? what is the condition for this to happen if
this happens?

This in the scenario where I am pushing MAAS machine learning model for threat detection.
JMeter tests over the endpoint tells me that the service (on localhost) increases response
time from 20ms to 1-2 seconds if there is an excessive load. 

How are YARN and Zookeeper making my service available and load-balanced? what is the criteria
to do so?

 

Also, this might be a (me) problem, but on my end, after the MAAS service has been deployed,
after several days, Stellar is not able to find the service anymore, even if it still registered
as a YARN application. If I restart Zookeeper the services gets back available on Stellar
again. Is this a known problem or is this a problem on my end?

Sorry for the lengthy question, but there is not much documentation on this.

 

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message