edgent-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dale LaBossiere <dml.apa...@gmail.com>
Subject Re: [disscuss] make TStream support groupBy operator?
Date Thu, 22 Mar 2018 17:06:21 GMT
Hi,  (I’ve been in my first month of retirement, yeah it would be helpful if other original
Edgent developers chimed in)

Edgent provides count or time based Windows of tuples.  A window inherently supports multiple
independently managed partitions by a key.   Continuous or batch aggregations can be performed
on each partition.

See TStream.last() and TWindow.  Additionally, see edgent.analytics.math3.* in particular
the javadoc for edgent.analytics.math3.Aggregations.

Hope that helps.

— Dale

> On Mar 20, 2018, at 5:28 AM, vino yang <yanghua1127@gmail.com> wrote:
> 
> Hi Chris,
> 
> My background is BigData (MapReduce, Spark, Flink, Kafka Stream) those data
> processing frameworks all provide the groupBy / keyBy operation and
> aggregation operator. It comes from traditional RDBMS. Edgent like a single
> JVM's Flink / Kafka Stream works on edges(gateway or IoT).
> They are similar with each other in some ways.
> 
> Vino yang.
> Thanks!
> 
> 2018-03-20 17:05 GMT+08:00 Christofer Dutz <christofer.dutz@c-ware.de>:
> 
>> Hi Vino,
>> 
>> unfortunately I can't contribute any opinion on this as I don't yet
>> understand the consequences.
>> I know that in an asynchronous event processing system some operations
>> that might be useful have to be sacrificed for the sake of asynchonisity.
>> 
>> For me Kafka Stream sort of feeling like the cloud-brother of Edgent, it
>> does seem to support groupBy.
>> 
>> Would be really cool if some of the formerly active people could at least
>> leave some comments on questions like this. You don't have to actually work
>> on things, but giving us new guys some guidance would be awesome.
>> 
>> I don't want to ruin thing you built over years, just because I'm not that
>> into the topic ... yet.
>> 
>> Chris
>> 
>> 
>> 
>> 
>> Am 16.03.18, 13:02 schrieb "Christofer Dutz" <christofer.dutz@c-ware.de>:
>> 
>>    I'm currently at a conference, so I can't be as responsive as I used
>> to be ... All will be back to normal next Tuesday ;-)
>> 
>>    Chris
>> 
>>    Outlook for Android<https://aka.ms/ghei36> herunterladen
>> 
>>    ________________________________
>>    From: vino yang <yanghua1127@gmail.com>
>>    Sent: Friday, March 16, 2018 2:26:10 AM
>>    To: dev@edgent.apache.org
>>    Subject: Re: [disscuss] make TStream support groupBy operator?
>> 
>>    Hi all,
>> 
>>    Anyone can give some opinion? Chris ? I think we should support some
>> reduce
>>    operation(aggregation function, such as max / avg / min sum) for both
>>    stream and windowed stream, these features based on the keyBy or
>> groupBy
>>    operation.
>> 
>>    Vino yang
>>    Thanks!
>> 
>>    2018-03-13 12:52 GMT+08:00 vino yang <yanghua1127@gmail.com>:
>> 
>>> Hi guys,
>>> 
>>> Does Edgent current support groupBy operator?
>>> 
>>> Vino yang
>>> Thanks.
>>> 
>> 
>> 
>> 


Mime
View raw message