Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cassandra.apache.org
Date: Wed, 4 Dec 2013 16:01:37 +0000 (UTC)
From: "Nicolas Favre-Felix (JIRA)" <jira@apache.org>
To: commits@cassandra.apache.org
Message-ID: <JIRA.12681552.1385555183071.70663.1386172897821@arcas>
In-Reply-To: <JIRA.12681552.1385555183071@arcas>
References: <JIRA.12681552.1385555183071@arcas>
Subject: [jira] [Commented] (CASSANDRA-6412) Custom creation and merge
 functions for user-defined column types
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/CASSANDRA-6412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13839028#comment-13839028 ] 

Nicolas Favre-Felix commented on CASSANDRA-6412:
------------------------------------------------

Thanks for the feedback, [~slebresne].

I like your suggestion to use user-defined types, this is definitely better than the home-made candlestick structure.
I also like that having fixed types with custom resolver makes it easier to write type-safe code with minimal changes to the Cassandra code base.

As you point out, we can use the same technique as for counter deletion. I understand that counter deletes are somewhat broken, and that columns with a custom resolver would suffer from a similar defect (CASSANDRA-2774).

I don't think that there is an easy solution to this problem; only deleting al CL.ALL would prevent old values from being merged with newer ones.

> Custom creation and merge functions for user-defined column types
> -----------------------------------------------------------------
>
>                 Key: CASSANDRA-6412
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-6412
>             Project: Cassandra
>          Issue Type: New Feature
>          Components: Core
>            Reporter: Nicolas Favre-Felix
>
> This is a proposal for a new feature, mapping custom types to Cassandra columns.
> These types would provide a creation function and a merge function, to be implemented in Java by the user.
> This feature relates to the concept of CRDTs; the proposal is to replicate "operations" on these types during write, to apply these operations internally during merge (Column.reconcile), and to also merge their values on read.
> The following operations are made possible without reading back any data:
> * MIN or MAX(value) for a column
> * First value for a column
> * Count Distinct
> * HyperLogLog
> * Count-Min
> And any composition of these too, e.g. a Candlestick type includes first, last, min, and max.
> The merge operations exposed by these types need to be commutative; this is the case for many functions used in analytics.
> This feature is incomplete without some integration with CASSANDRA-4775 (Counters 2.0) which provides a Read-Modify-Write implementation for distributed counters. Integrating custom creation and merge functions with new counters would let users implement complex CRDTs in Cassandra, including:
> * Averages & related (sum of squares, standard deviation)
> * Graphs
> * Sets
> * Custom registers (even with vector clocks)
> I have a working prototype with implementations for min, max, and Candlestick at https://github.com/acunu/cassandra/tree/crdts - I'd appreciate any feedback on the design and interfaces.


--
This message was sent by Atlassian JIRA
(v6.1#6144)