edgent-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dale LaBossiere <dml.apa...@gmail.com>
Subject Re: Small (memory) footprint specifics
Date Tue, 02 Jan 2018 17:12:50 GMT
Sorry, I don’t have any sizing numbers to offer.  Don’t know if others on the list have
any experiences they’d like to share.

As you expect, which connectors are used would likely have a big influence on the size.  In
many cases we’re at the mercy of an underlying 3rd party connector client library (e.g.,
MQTT, Kafka).

I would hope that you wouldn’t find it too difficult / time consuming to construct a trivial
prototype app for your environment.  Then you can measure it… and hopefully report the results
here :-)

Note, DevelopmentSample is a poor choice for another reason: use of the DevelopmentProvider,
which supplies the Edgent Console for that app, isn’t expected to be in a deployed/production
Edgent application.  Stick to the DirectProvider or IotProvider depending on your needs.

There are samples for several connectors.  There isn’t one for the HttpStreams connector
[1] but its method doc has some sample usage code.

We don’t have a JMX connector but that shouldn’t be a big impediment.  E.g., a “connector”
can be as simple as a function that gets a data item every time it’s called. You may have
already noticed [2] (which also points to some example code using ElasticSearch), and [3].
 If you end up developing a more formal JMX connector please consider contributing it!

e.g., a prototype app something along the lines of

public static void main(String[] args) throws Exception {
    DirectProvider p = new DirectProvider();
    Topology top = p.newTopology();
    final String url = “http://httpbin.org/put”;

    TStream<JsonObject> myJmxStream = topology.poll(
							() -> someMethodThatGetsTheNextDataAndReturnsItAsJson(),
							1, TimeUnit.SECONDS );
    TStream<JsonObject> rc = HttpStreams.putJson(myJmxStream,
							HttpClients::noAuthentication, t -> url, t -> t);
    // myJmxStream.sink(t -> System.out.println(“Data: “ + t));

    p.submit(top); 
}

[1] http://edgent.apache.org/javadoc/latest/org/apache/edgent/connectors/http/HttpStreams.html
[2] http://edgent.apache.org/docs/power-of-edgent.html <http://edgent.apache.org/docs/power-of-edgent.html>
[3] http://edgent.apache.org/recipes/recipe_writing_a_connector.html <http://edgent.apache.org/recipes/recipe_writing_a_connector.html>

Hope that helps!        

— Dale

> On Dec 29, 2017, at 5:05 PM, Christofer Dutz <christofer.dutz@c-ware.de> wrote:
> 
> Unfortunately I can't provide you with such data as I am just getting started in using
Edgent in real applications. Maybe Dale might help you with that.
> 
> Chris
> 
> Outlook for Android<https://aka.ms/ghei36> herunterladen
> 
> ________________________________
> From: Otis Gospodnetić <otis.gospodnetic@gmail.com>
> Sent: Friday, December 29, 2017 10:36:21 PM
> To: dev@edgent.apache.org
> Subject: Re: Small (memory) footprint specifics
> 
> Hi Chris,
> 
> Thanks for the quick reply!
> 3.9 MB heap (was the RSS also cca 4 MB?) is much smaller than I anticipated
> given this is running in the JVM.  What's the trick? :)
> 
> I assume that DevelopmentSample app is the one from
> https://github.com/apache/incubator-edgent-samples/tree/develop/topology/src/main/java/org/apache/edgent/samples/topology
> ?
> 
> How much bigger might the memory footprint be if one ran a more realistic
> app, say something that:
> * get data from JMX (so RMI involved) or calls some HTTP API that returns
> JSON (so parsing needed)
> * labels/tags data (like in the
> https://github.com/apache/incubator-edgent-samples/blob/develop/topology/src/main/java/org/apache/edgent/samples/topology/StreamTags.java
> )
> 
> ... but still no windowed analysis on the agent side.
> 
> Are we then talking 10-20-30+ MB?
> 
> Thanks,
> Otis
> --
> Monitoring - Log Management - Alerting - Anomaly Detection
> Solr & Elasticsearch Consulting Support Training - http://sematext.com/
> 
> 
> On Fri, Dec 29, 2017 at 2:38 PM, Christofer Dutz <christofer.dutz@c-ware.de>
> wrote:
> 
>> Hi Otis,
>> 
>> So, I just ran the DevelopmentSample application in JProfiler and it took
>> a minimum of 3,9MB of heap … This however is neither a very representative
>> Example, but it should demonstrate that you don’t need a lot of memory to
>> run an Edgent application. I guess it also depends highly on the type of
>> operations. If you use a lot of Windowed analysis, I would expect the
>> memory requirements to be a lot higher than when using only pure stream
>> operations that don’t require keeping events in memory.
>> 
>> Chris
>> 
>> Am 29.12.17, 17:29 schrieb "Otis Gospodnetić" <otis.gospodnetic@gmail.com
>>> :
>> 
>>    Hi,
>> 
>>    This is from the FAQ:
>> 
>>    *Edgent is designed for the edge. It has a small footprint, suitable
>> for
>>    running on constrained devices.*
>> 
>>    Can anyone share some specifics of that "small footprint" part in
>> terms of
>>    memory?  How small of a footprint can Edgent have?
>> 
>>    Thanks,
>>    Otis
>>    --
>>    Monitoring - Log Management - Alerting - Anomaly Detection
>>    Solr & Elasticsearch Consulting Support Training -
>> http://sematext.com/
>> 
>> 
>> 


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message