cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Benjamin Roth <benjamin.r...@jaumo.com>
Subject Re: implementing a 'sorted set' on top of cassandra
Date Sat, 14 Jan 2017 07:54:45 GMT
If your proposed solution is crazy depends on your needs :)
It sounds like you can live with not-realtime data. So it is ok to cache
it. Why preproduce the results if you only need 5% of them? Why not use
redis as a cache with expiring sorted sets that are filled on demand from
cassandra partitions with counters?
So redis has much less to do and can scale much better. And you are not
limited on keeping all data in ram as cache data is volatile and can be
evicted on demand.
If this is effective also depends on the size of your sets. CS wont be able
to sort them by score for you, so you will have to load the complete set to
redis for caching and / or do sorting in your app on demand. This certainly
won't work out well with sets with millions of entries.

2017-01-13 23:14 GMT+01:00 Mike Torra <mtorra@demandware.com>:

> We currently use redis to store sorted sets that we increment many, many
> times more than we read. For example, only about 5% of these sets are ever
> read. We are getting to the point where redis is becoming difficult to
> scale (currently at >20 nodes).
>
> We've started using cassandra for other things, and now we are
> experimenting to see if having a similar 'sorted set' data structure is
> feasible in cassandra. My approach so far is:
>
>    1. Use a counter CF to store the values I want to sort by
>    2. Periodically read in all key/values in the counter CF and sort in
>    the client application (~every five minutes or so)
>    3. Write back to a different CF with the ordered keys I care about
>
> Does this seem crazy? Is there a simpler way to do this in cassandra?
>



-- 
Benjamin Roth
Prokurist

Jaumo GmbH · www.jaumo.com
Wehrstraße 46 · 73035 Göppingen · Germany
Phone +49 7161 304880-6 · Fax +49 7161 304880-1
AG Ulm · HRB 731058 · Managing Director: Jens Kammerer

Mime
View raw message