edgent-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christofer Dutz <christofer.d...@c-ware.de>
Subject Re: USP Apache Edgent
Date Tue, 07 Aug 2018 07:26:55 GMT
Hi Julian,



I would really like to give you input about the internals and best practices of Edgent, however
I can't :-(



I started working on Edgent after hearing from it in Miami and had been concentrating on fixing
the build system (Did the Ant->Maven conversion). So I really understand 100% of the build
system, but shortly after releasing 1.2.0 the others seem to have shifted their priorities.
So right now deep knowledge of Edgent is scarce on this list. I am trying to dig in a little
deeper every day, but as you know ... most of my time currently goes into PLC4X. But for me
Edgent is THE tool for writing PLC4X Applications and I'm not going to abandon it.



Interesting that you mention a drop-in replacement of Kafka, as I was planning on suggesting
this on this list. As for me there is a direct correlation between Kafka Streams and Edgent
Streams, I was planning on suggesting to streamline the naming of things as well as the syntax.
So someone able to write applications for one of the two will be able to achieve the same
in the other.



Chris







Am 07.08.18, 08:39 schrieb "Julian Feinauer" <j.feinauer@pragmaticminds.de>:



    Hey Chris,

    

    

    

    hello to you on this list and thanks for your response.

    

    I get your point and I agree that Akka streams is very complex which is especially bad
for simple use cases.

    

    But, as you also say, Edgent and Akka streams seem to be the first options for streaming
in situations where one has limited resources (which is also often times the situation for
us).

    

    

    

    What I liked about Edgent when I first looked into it (must be 2 years ago, I think shortly
after it was renamed from Quartz) I liked the command & control approach but up till now
I'm not able to understand the concept (it instruments something wich MBeans and JMX it seems
but I can't figure it out). Do you have any hints for "good" documentation on that or at least
some code samples which help to get the principle?

    

    

    

    Another point I am currently missing in the IoT Landscape is a (logical) drop in replacement
for something like Apache Kafka. We use it usually for two reasons. First, the obvious decoupling
from data generation and processing (and the possibility to backpressure) and second as a
message store (in the concept of Event Sourcing) to allow for later reprocessing.

    

    Stripping it down, without all the distributed stuff Kafka is "just" an ordered queue
(as Jay Kreps likes to emphasize also). So in the context of stream processing it would love
to have a Sink / Source with semantics similar to kafkas (producer, consumer, topics, offset
management) which is backed by a simple append only file. One interesting Implementation of
such a queue (without the exact semantics, I think, is e.g., [1]).

    

    Is there something similar in Edgent currently, I did not observe anything like that.

    

    

    

    Would you think that this is something which would fit in Edgents scope as IoT Streaming
project?

    

    

    

    Best

    

    Julian

    

    

    

    [1] https://github.com/OpenHFT/Chronicle-Queue

    

    

    

    Am 06.08.18, 23:32 schrieb "Christofer Dutz" <christofer.dutz@c-ware.de>:

    

    

    

        Hi Julian,

    

        

    

        

    

        

    

        welcome here on the Edgent list :-)

    

        

    

        

    

        

    

        Well I can't speak for the rest of the team, but for me Edgent is currently sort of
the following:

    

        

    

        - For me Edgent is the ideal entry point for any sort of IoT system 

    

        

    

        - I like to compare it with Kakfa-Connect, but without the cloud (ok we can read and
write to clouds, but Edgent is not a cloud based system)

    

        

    

        

    

        

    

        Especially if you want to operate in constrained environments, such as: 

    

        

    

        - Limited processing power

    

        

    

        - Low energy consumption

    

        

    

        - Low memory consumption

    

        

    

        - Narrow/Unreliable/High-Latency network connectivity

    

        

    

        Then is where you can cross all the cloud based streaming frameworks off your list.

    

        

    

        

    

        

    

        Regarding Akka Streams ... well the presentations I did some time ago with my little
Rapiro Robot, were based on Akka Streams ... that was why I fell in love with Edgent when
I heard about it at ApacheCon NA in Miami. For me, Akka Streams totally fries my neurons.
It's far too complicated. And as soon as you want to work with Fan-In/Fan-Out and Backpressure
... well ... I thought it was nuts ... but I have to admit that I'm no Akka or Scala pro,
so please don't treat this assumption as a fact. It's just my opinion. 

    

        

    

        

    

        

    

        Does that answer your question? If not ... feel free to ask :)

    

        

    

        

    

        

    

        Chris

    

        

    

        

    

        

    

        

    

        

    

        Am 06.08.18, 13:50 schrieb "Julian Feinauer" <j.feinauer@pragmaticminds.de>:

    

        

    

        

    

        

    

            Hi all,

    

        

    

            

    

        

    

            this is my first post on the Apache Edgent mailing list and first, let me say,
that I’m quite impressed by all that has been done with regards to IoT and Edge Processing
in this project.

    

        

    

            As I observe the project for a long time now and we start to see many use cases
with our customers we decided to get deeper in touch with the community.

    

        

    

            

    

        

    

            When reasoning about the architecture of a new system for our new costumers including
a lot of “IoT” communication we obviously came by apache edgent.

    

        

    

            Can you say, what in your opinion the unique selling point of edgent is?

    

        

    

            Is the focus on the interaction with Cloud Based IoT Services or do you see the
low footprint stream processing also as one advantage of edgent?

    

        

    

            I tried to get a really good overview over the remote control capabilities and
the MXBean integration, but I think I’m not at it, yet.

    

        

    

            

    

        

    

            We looked at different stream processing platforms like Flink, Spark (which are
everything except low footprint) but also Akka streams which seems to fit very well with “low
footprint”.

    

        

    

            So do you see Akka streams as an alternative to Edgent when one only uses a “Simple”
Topology with a DirectProvider and no integration to Remote Services or Control and Command
Servers or anything?

    

        

    

            

    

        

    

            I would love to hear your opinion on that!

    

        

    

            

    

        

    

            Best

    

        

    

            Julian

    

        

    

            

    

        

    

            pragmatic minds GmbH

    

        

    

            

    

        

    

        

    

        

    

    

    


Mime
View raw message