cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nate McCall <n...@thelastpickle.com>
Subject Re: Order by for aggregated values
Date Wed, 07 Jun 2017 02:27:57 GMT
>
>
> My application is a real-time application. It monitors devices in the
> network and displays the top N devices for various parameters averaged over
> a time period. A query may involve anywhere from 10 to 50k devices, and
> anywhere from 5 to 2000 intervals. We expect a query to take less than 2
> seconds.
>
>
>
> My impression was that Spark is aimed at larger scale analytics.
>
>
>
> I am ok with the limitation on “group by”. I am intending to use async
> queries and token-aware load balancing to partition the query and execute
> it in parallel on each node.
>
>
>

This sounds a lot more like a use case for a streaming system (run in
parallel with Cassandra).

Apache Flink might be one avenue to explore - their Cassandra integration
works fine, btw.

A lot of folks are doing similar things with Apache Beam as well as it has
quite an elegant paradigm for the use case you describe, particularly if
you need to combine batching with streaming. (FYI, their "CassandraIO" is
about to be merged in master:
https://github.com/apache/beam/pull/592#issuecomment-306618338).


-- 
-----------------
Nate McCall
Wellington, NZ
@zznate

CTO
Apache Cassandra Consulting
http://www.thelastpickle.com

Mime
View raw message