cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kevin Burton <bur...@spinn3r.com>
Subject Re: Efficiently filtering results directly in CS
Date Sat, 09 Apr 2016 00:22:55 GMT
Ha..  Yes... C*...  I guess I need something like coprocessors in bigtable.


On Fri, Apr 8, 2016 at 1:49 AM, vincent gromakowski <
vincent.gromakowski@gmail.com> wrote:

> c* I suppose
>
> 2016-04-07 19:30 GMT+02:00 Jonathan Haddad <jon@jonhaddad.com>:
>
>> What is CS?
>>
>> On Thu, Apr 7, 2016 at 10:03 AM Kevin Burton <burton@spinn3r.com> wrote:
>>
>>> I have a paging model whereby we stream data from CS by fetching 'pages'
>>> thereby reading (sequentially) entire datasets.
>>>
>>> We're using the bucket approach where we write data for 5 minutes, then
>>> we can just fetch the bucket for that range.
>>>
>>> Our app now has TONS of data and we have a piece of middleware that
>>> filters it based on the client requests.
>>>
>>> So if they only want english they just get english and filter away about
>>> 60% of our data.
>>>
>>> but it doesn't support condition pushdown.  So ALL this data has to be
>>> sent from our CS boxes to our middleware and filtered there (wasting a lot
>>> of network IO).
>>>
>>> Is there away (including refactoring the code) that I could push this
>>> this into CS?  Maybe some way I could discovery the CS topology and put
>>> daemons on each of our CS boxes and fetch from CS directly (doing the
>>> filtering there).
>>>
>>> Thoughts?
>>>
>>> --
>>>
>>> We’re hiring if you know of any awesome Java Devops or Linux Operations
>>> Engineers!
>>>
>>> Founder/CEO Spinn3r.com
>>> Location: *San Francisco, CA*
>>> blog: http://burtonator.wordpress.com
>>> … or check out my Google+ profile
>>> <https://plus.google.com/102718274791889610666/posts>
>>>
>>>
>


-- 

We’re hiring if you know of any awesome Java Devops or Linux Operations
Engineers!

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
<https://plus.google.com/102718274791889610666/posts>

Mime
View raw message