edgent-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Julian Feinauer <j.feina...@pragmaticminds.de>
Subject Re: USP Apache Edgent
Date Tue, 07 Aug 2018 06:39:23 GMT
Hey Chris,

hello to you on this list and thanks for your response.

I get your point and I agree that Akka streams is very complex which is especially bad for
simple use cases.

But, as you also say, Edgent and Akka streams seem to be the first options for streaming in
situations where one has limited resources (which is also often times the situation for us).

What I liked about Edgent when I first looked into it (must be 2 years ago, I think shortly
after it was renamed from Quartz) I liked the command & control approach but up till now
I'm not able to understand the concept (it instruments something wich MBeans and JMX it seems
but I can't figure it out). Do you have any hints for "good" documentation on that or at least
some code samples which help to get the principle?

Another point I am currently missing in the IoT Landscape is a (logical) drop in replacement
for something like Apache Kafka. We use it usually for two reasons. First, the obvious decoupling
from data generation and processing (and the possibility to backpressure) and second as a
message store (in the concept of Event Sourcing) to allow for later reprocessing.

Stripping it down, without all the distributed stuff Kafka is "just" an ordered queue (as
Jay Kreps likes to emphasize also). So in the context of stream processing it would love to
have a Sink / Source with semantics similar to kafkas (producer, consumer, topics, offset
management) which is backed by a simple append only file. One interesting Implementation of
such a queue (without the exact semantics, I think, is e.g., [1]).

Is there something similar in Edgent currently, I did not observe anything like that.

Would you think that this is something which would fit in Edgents scope as IoT Streaming project?



[1] https://github.com/OpenHFT/Chronicle-Queue

Am 06.08.18, 23:32 schrieb "Christofer Dutz" <christofer.dutz@c-ware.de>:

    Hi Julian,




    welcome here on the Edgent list :-)




    Well I can't speak for the rest of the team, but for me Edgent is currently sort of the


    - For me Edgent is the ideal entry point for any sort of IoT system 


    - I like to compare it with Kakfa-Connect, but without the cloud (ok we can read and write
to clouds, but Edgent is not a cloud based system)




    Especially if you want to operate in constrained environments, such as: 


    - Limited processing power


    - Low energy consumption


    - Low memory consumption


    - Narrow/Unreliable/High-Latency network connectivity


    Then is where you can cross all the cloud based streaming frameworks off your list.




    Regarding Akka Streams ... well the presentations I did some time ago with my little Rapiro
Robot, were based on Akka Streams ... that was why I fell in love with Edgent when I heard
about it at ApacheCon NA in Miami. For me, Akka Streams totally fries my neurons. It's far
too complicated. And as soon as you want to work with Fan-In/Fan-Out and Backpressure ...
well ... I thought it was nuts ... but I have to admit that I'm no Akka or Scala pro, so please
don't treat this assumption as a fact. It's just my opinion. 




    Does that answer your question? If not ... feel free to ask :)










    Am 06.08.18, 13:50 schrieb "Julian Feinauer" <j.feinauer@pragmaticminds.de>:




        Hi all,




        this is my first post on the Apache Edgent mailing list and first, let me say, that
I’m quite impressed by all that has been done with regards to IoT and Edge Processing in
this project.


        As I observe the project for a long time now and we start to see many use cases with
our customers we decided to get deeper in touch with the community.




        When reasoning about the architecture of a new system for our new costumers including
a lot of “IoT” communication we obviously came by apache edgent.


        Can you say, what in your opinion the unique selling point of edgent is?


        Is the focus on the interaction with Cloud Based IoT Services or do you see the low
footprint stream processing also as one advantage of edgent?


        I tried to get a really good overview over the remote control capabilities and the
MXBean integration, but I think I’m not at it, yet.




        We looked at different stream processing platforms like Flink, Spark (which are everything
except low footprint) but also Akka streams which seems to fit very well with “low footprint”.


        So do you see Akka streams as an alternative to Edgent when one only uses a “Simple”
Topology with a DirectProvider and no integration to Remote Services or Control and Command
Servers or anything?




        I would love to hear your opinion on that!










        pragmatic minds GmbH






View raw message