cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From vincent gromakowski <vincent.gromakow...@gmail.com>
Subject Re: Efficiently filtering results directly in CS
Date Sat, 09 Apr 2016 06:55:55 GMT
spark over c* can pushdown lots of things (basic filter or where clause to
more advanced semi join)

2016-04-09 3:54 GMT+02:00 kurt Greaves <kurt@instaclustr.com>:

> If you're using C* 3.0 you can probably achieve this with UDFs.
> http://www.planetcassandra.org/blog/user-defined-functions-in-cassandra-3-0/
>
> On 9 April 2016 at 00:22, Kevin Burton <burton@spinn3r.com> wrote:
>
>> Ha..  Yes... C*...  I guess I need something like coprocessors in
>> bigtable.
>>
>> On Fri, Apr 8, 2016 at 1:49 AM, vincent gromakowski <
>> vincent.gromakowski@gmail.com> wrote:
>>
>>> c* I suppose
>>>
>>> 2016-04-07 19:30 GMT+02:00 Jonathan Haddad <jon@jonhaddad.com>:
>>>
>>>> What is CS?
>>>>
>>>> On Thu, Apr 7, 2016 at 10:03 AM Kevin Burton <burton@spinn3r.com>
>>>> wrote:
>>>>
>>>>> I have a paging model whereby we stream data from CS by fetching
>>>>> 'pages' thereby reading (sequentially) entire datasets.
>>>>>
>>>>> We're using the bucket approach where we write data for 5 minutes,
>>>>> then we can just fetch the bucket for that range.
>>>>>
>>>>> Our app now has TONS of data and we have a piece of middleware that
>>>>> filters it based on the client requests.
>>>>>
>>>>> So if they only want english they just get english and filter away
>>>>> about 60% of our data.
>>>>>
>>>>> but it doesn't support condition pushdown.  So ALL this data has to be
>>>>> sent from our CS boxes to our middleware and filtered there (wasting
a lot
>>>>> of network IO).
>>>>>
>>>>> Is there away (including refactoring the code) that I could push this
>>>>> this into CS?  Maybe some way I could discovery the CS topology and put
>>>>> daemons on each of our CS boxes and fetch from CS directly (doing the
>>>>> filtering there).
>>>>>
>>>>> Thoughts?
>>>>>
>>>>> --
>>>>>
>>>>> We’re hiring if you know of any awesome Java Devops or Linux
>>>>> Operations Engineers!
>>>>>
>>>>> Founder/CEO Spinn3r.com
>>>>> Location: *San Francisco, CA*
>>>>> blog: http://burtonator.wordpress.com
>>>>> … or check out my Google+ profile
>>>>> <https://plus.google.com/102718274791889610666/posts>
>>>>>
>>>>>
>>>
>>
>>
>> --
>>
>> We’re hiring if you know of any awesome Java Devops or Linux Operations
>> Engineers!
>>
>> Founder/CEO Spinn3r.com
>> Location: *San Francisco, CA*
>> blog: http://burtonator.wordpress.com
>> … or check out my Google+ profile
>> <https://plus.google.com/102718274791889610666/posts>
>>
>>
>
>
> --
> Kurt Greaves
> kurt@instaclustr.com
> www.instaclustr.com
>

Mime
View raw message