Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 8C2C810A5C for ; Tue, 25 Feb 2014 19:12:17 +0000 (UTC) Received: (qmail 28229 invoked by uid 500); 25 Feb 2014 19:12:14 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 28149 invoked by uid 500); 25 Feb 2014 19:12:14 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 28140 invoked by uid 99); 25 Feb 2014 19:12:14 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 25 Feb 2014 19:12:14 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of mdykman@gmail.com designates 209.85.216.180 as permitted sender) Received: from [209.85.216.180] (HELO mail-qc0-f180.google.com) (209.85.216.180) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 25 Feb 2014 19:12:10 +0000 Received: by mail-qc0-f180.google.com with SMTP id i17so11440011qcy.39 for ; Tue, 25 Feb 2014 11:11:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=Qv1WkMXCBjNAm64brWggPwf/UaJMLk6I9jGG7bp/Cy8=; b=tD4/Wvh/ZJgJvka4f1ZTo5cdCFBT3klZwux6PUGI0dVGkq5bZxlRaYjYFpqyJ+z8Gu vT3+pXYkDeaRzhk91V5d3V4WX96So9YMRn/D7Rzn+maZ9GsNf+WJ58Yfk72THtfxguqM 0NaRn8j7jn50KLtyd9ixLyjZ5JsDpj9q0kUFz8AspPtKMmv3lD8jBIQiT61tL0ZsTeIh JcbZlpG8nQeQblcKimyyZEju1h/yPWgColWMIHuF+qnwpkhIbyZFJ/kS2v3ZRigC5p1F GFsESOSwdYn4Rp1HWVd7c3EALEkVywu+66kKvYdlHCkv5O0R90ZXEN6wElxmk2/pMzp/ rykw== MIME-Version: 1.0 X-Received: by 10.224.5.136 with SMTP id 8mr2586634qav.42.1393355509522; Tue, 25 Feb 2014 11:11:49 -0800 (PST) Received: by 10.140.96.104 with HTTP; Tue, 25 Feb 2014 11:11:49 -0800 (PST) Received: by 10.140.96.104 with HTTP; Tue, 25 Feb 2014 11:11:49 -0800 (PST) In-Reply-To: References: Date: Tue, 25 Feb 2014 14:11:49 -0500 Message-ID: Subject: Re: How should clients handle the user defined types in 2.1? From: Michael Dykman To: user@cassandra.apache.org Content-Type: multipart/alternative; boundary=001a11c2c59a30e95904f33fdc50 X-Virus-Checked: Checked by ClamAV on apache.org --001a11c2c59a30e95904f33fdc50 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Please do. I too am working at a driver implementation and would be delighted to be spared the research. On Feb 25, 2014 11:29 AM, "Theo Hultberg" wrote: > thanks for the high level description of the format, I'll see if I can > make a stab at implementing support for custom types now. > > and maybe I should take all of the reverse engineering I've done of the > type encoding and decoding and send a pull request for the protocol spec, > or write an appendix. > > T# > > > On Tue, Feb 25, 2014 at 12:10 PM, Sylvain Lebresne = wrote: > >> >>> Is there any documentation on how CQL clients should handle the new use= r >>> defined types coming in 2.1? There's nothing in the protocol specificat= ion >>> on how to handle custom types as far as I can see. >>> >> >> Can't say there is much documentation so far for that. As for the spec, >> it was written in a time where user defined types didn't existed and so = as >> far as the protocol is concerned so far, user defined types are handled = by >> the protocol as a "custom type", i.e the full internal class is returned= . >> And so ... >> >> >>> >>> For example, I tried creating the "address" type from the description o= f >>> CASSANDRA-5590, and this is how its metadata looks (the metadata for a >>> query contains a column with a custom type and this is the description = of >>> it): >>> >>> >>> org.apache.cassandra.db.marshal.UserType(user_defined_types,61646472657= 373,737472656574:org.apache.cassandra.db.marshal.UTF8Type,63697479:org.apac= he.cassandra.db.marshal.UTF8Type,7a69705f636f6465:org.apache.cassandra.db.m= arshal.Int32Type,70686f6e6573:org.apache.cassandra.db.marshal.SetType(org.a= pache.cassandra.db.marshal.UTF8Type)) >>> >>> Is the client supposed to parse that description, and in that case how? >>> >> >> ... yes, for now you're supposed to parse that description. Which is not >> really much documented outside of looking up the Cassandra code, but I c= an >> tell you that the first parameter of the UserType is the keyspace name t= he >> type has been defined in, the second is the type name hex encoded, and t= he >> rest is list of fields and their type. Each field name is hex encoded an= d >> separated from it's type by ':'. And that's about it. >> >> We will introduce much shorted definitions in the next iteration of the >> native protocol, but it's yet unclear when that will happen. >> >> -- >> Sylvain >> >> >> > --001a11c2c59a30e95904f33fdc50 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable

Please do.=A0 I too am working at a driver implementation and would be d= elighted to be spared the research.

On Feb 25, 2014 11:29 AM, "Theo Hultberg&qu= ot; <theo@iconara.net> wrote:=
thanks for the high level description of the format, I'= ;ll see if I can make a stab at implementing support for custom types now.<= div>
and maybe I should take all of the reverse engineering I= 've done of the type encoding and decoding and send a pull request for = the protocol spec, or write an appendix.

T#


On Tue, Feb 25, 2014 at 12:10 PM, Sylvain Lebresne <sylvain@datastax.com> wrote:
=

Is there any documentation on how CQL clients should handle the new user de= fined types coming in 2.1? There's nothing in the protocol specificatio= n on how to handle custom types as far as I can see.

Can't say there is much documentation so far = for that. As for the spec, it was written in a time where user defined type= s didn't existed and so as far as the protocol is concerned so far, use= r defined types are handled by the protocol as a "custom type", i= .e the full internal class is returned. And so ...
=A0

For ex= ample, I tried creating the "address" type from the description o= f CASSANDRA-5590, and this is how its metadata looks (the metadata for a qu= ery contains a column with a custom type and this is the description of it)= :

org.apache.cassandra.d= b.marshal.UserType(user_defined_types,61646472657373,737472656574:org.apach= e.cassandra.db.marshal.UTF8Type,63697479:org.apache.cassandra.db.marshal.UT= F8Type,7a69705f636f6465:org.apache.cassandra.db.marshal.Int32Type,70686f6e6= 573:org.apache.cassandra.db.marshal.SetType(org.apache.cassandra.db.marshal= .UTF8Type))

=
Is the client su= pposed to parse that description, and in that case how?

... yes, for now you're supposed to parse tha= t description. Which is not really much documented outside of looking up th= e Cassandra code, but I can tell you that the first parameter of the UserTy= pe is the keyspace name the type has been defined in, the second is the typ= e name hex encoded, and the rest is list of fields and their type. Each fie= ld name is hex encoded and separated from it's type by ':'. And= that's about it.
=A0
We will introduce much shorted definitions in the ne= xt iteration of the native protocol, but it's yet unclear when that wil= l happen.

--
Sylvain



--001a11c2c59a30e95904f33fdc50--