Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 591CC107EC for ; Tue, 25 Feb 2014 12:51:12 +0000 (UTC) Received: (qmail 77005 invoked by uid 500); 25 Feb 2014 12:51:09 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 76791 invoked by uid 500); 25 Feb 2014 12:51:08 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 76774 invoked by uid 99); 25 Feb 2014 12:51:07 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 25 Feb 2014 12:51:07 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of woolfel@gmail.com designates 209.85.220.176 as permitted sender) Received: from [209.85.220.176] (HELO mail-vc0-f176.google.com) (209.85.220.176) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 25 Feb 2014 12:51:03 +0000 Received: by mail-vc0-f176.google.com with SMTP id la4so7051625vcb.21 for ; Tue, 25 Feb 2014 04:50:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=DbSN47ZhrMRklTn2jGzWL3R7Qw9oO02SLmf10/IJgEI=; b=yq/8l35HYuqGaCHjfrse7gfiqJZpWQBt87oaaiRykEa/Sb7IX04r9cqCNdE/sTdLtH cxaLZGAE9WqnsUL47ibjaBltsm9Aac520jx+qdH1bzvZ/b4NYZwzmxcPuRTLcJWXQkgB rgCt1peTYZ/h4QYvFEu5N+aV7iS9SRx4GH1Y9lwASO/aYk1StNTTmrUfCutPrqx3wv+s nhsCszEr0Hsu/Ye/u1cgh489IRBU+JmEZ4L2UrJGSMVuAfZ4MMR/KZXO7+d0o1D7Tloz /kDVKWFv54+mvIRsb9DJveOXHBIPb5qsiBwPWgUIHMhp300jdTENo7hy3dI+XcCEfQ4i di7g== MIME-Version: 1.0 X-Received: by 10.52.179.198 with SMTP id di6mr823430vdc.7.1393332642769; Tue, 25 Feb 2014 04:50:42 -0800 (PST) Received: by 10.221.17.198 with HTTP; Tue, 25 Feb 2014 04:50:42 -0800 (PST) In-Reply-To: References: <20140224184009.58518b29@shy.leonerd.org.uk> <20140224190214.009083a4@shy.leonerd.org.uk> <20140224194307.7ad256cd@shy.leonerd.org.uk> Date: Tue, 25 Feb 2014 07:50:42 -0500 Message-ID: Subject: Re: CQL decimal encoding From: Peter Lin To: "user@cassandra.apache.org" Content-Type: multipart/alternative; boundary=bcaec51a77fa39f12a04f33a89b1 X-Virus-Checked: Checked by ClamAV on apache.org --bcaec51a77fa39f12a04f33a89b1 Content-Type: text/plain; charset=ISO-8859-1 if I have time this week, I'll try to make a patch for the spec. Can't promise I can get to it this week, but having come across this issue with FluentCassandra, I'd like to help others avoid it. On Tue, Feb 25, 2014 at 5:38 AM, Sylvain Lebresne wrote: > On Mon, Feb 24, 2014 at 8:50 PM, Theo Hultberg wrote: > >> I don't know if it's by design or if it's by oversight that the data >> types aren't part of the binary protocol specification. >> > > The honest answer is, no-one took the time to write that down properly and > include it in the spec. My small excuse for initially skipping it in the > spec is that the CQL data type encodings are really not different from what > we have had in thrift since forever, so that there is already many driver > in a lot of language out there that have encoding/decoding functions for > them that can be looked at. But if someone find some time to gather all > those encoding and provides a patch for the spec with them, that would > definitively be much appreciated. Cassandra is an open-source software, > everyone can contribute, and that does not exclude documentation and > specifications. > > Regarding the decimal, it does uses 4 bytes for the scale and the rest for > the bytes of the unscaled value (i.e. a variable length integer) as Peter > mentioned. the actual value being the "unscaled value" * 10^-"scale" > (internally C* really just use the following BigDecimal ctor: > http://docs.oracle.com/javase/7/docs/api/java/math/BigDecimal.html#BigDecimal(java.math.BigInteger, > int)). > > -- > Sylvain > > > >> I had to reverse engineer how to encode and decode all of them for the >> Ruby driver. There were definitely a few bugs in the first few versions >> that could have been avoided if there was a specification available. >> >> T# >> >> >> On Mon, Feb 24, 2014 at 8:43 PM, Paul "LeoNerd" Evans < >> leonerd@leonerd.org.uk> wrote: >> >>> On Mon, 24 Feb 2014 19:14:48 +0000 >>> Ben Hood <0x6e6562@gmail.com> wrote: >>> >>> > So I have a question about the encoding of 0: \x00\x00\x00\x00\x00. >>> >>> The first four octets are the decimal shift (0), and the remaining ones >>> (one in this case) encode a varint - 0 in this case. So it's >>> >>> 0 * 10**0 >>> >>> literally zero. >>> >>> Technically the decimal shift matters not for zero - any four bytes >>> could be given as the shift, ending in \x00, but 0 is the simplest. >>> >>> -- >>> Paul "LeoNerd" Evans >>> >>> leonerd@leonerd.org.uk >>> ICQ# 4135350 | Registered Linux# 179460 >>> http://www.leonerd.org.uk/ >>> >> >> > --bcaec51a77fa39f12a04f33a89b1 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable

if I have time this week, I'll try to m= ake a patch for the spec. Can't promise I can get to it this week, but = having come across this issue with FluentCassandra, I'd like to help ot= hers avoid it.


On Tue,= Feb 25, 2014 at 5:38 AM, Sylvain Lebresne <sylvain@datastax.com>= ; wrote:
=
On Mon, Feb 24, 2014 at 8:50 PM,= Theo Hultberg <theo@iconara.net> wrote:
I don't know if= it's by design or if it's by oversight that the data types aren= 9;t part of the binary protocol specification.

The honest answer is, no-one took th= e time to write that down properly and include it in the spec. My small exc= use for initially skipping it in the spec is that the CQL data type encodin= gs are really not different from what we have had in thrift since forever, = so that there is already many driver in a lot of language out there that ha= ve encoding/decoding functions for them that can be looked at. But if someo= ne find some time to gather all those encoding and provides a patch for the= spec with them, that would definitively be much appreciated. Cassandra is = an open-source software, everyone can contribute, and that does not exclude= documentation and specifications.

Regarding the decimal, it does uses 4 bytes for the sca= le and the rest for the bytes of the unscaled value (i.e. a variable length= integer) as Peter mentioned. the actual value being the "unscaled val= ue" * 10^-"scale" (internally C* really just use the followi= ng BigDecimal ctor: =A0http://docs.oracle.com/javase/7/docs/api/java/math/BigDecimal.html#BigDec= imal(java.math.BigInteger, int)).

--
Sylvain

=A0
I had to reverse engineer how to encode and decode all of = them for the Ruby driver. There were definitely a few bugs in the first few= versions that could have been avoided if there was a specification availab= le.

T#


On Mon, Feb 24, 2014 at= 8:43 PM, Paul "LeoNerd" Evans <leonerd@leonerd.org.uk&= gt; wrote:
On Mon, 24 Feb 2014 19:14:48 +0000
Ben Hood <0x6e65= 62@gmail.com> wrote:

> So I have a question about the encoding of 0: \x00\x00\x00\x00\x00.
The first four octets are the decimal shift (0), and the remaining on= es
(one in this case) encode a varint - 0 in this case. So it's

=A0 0 * 10**0

literally zero.

Technically the decimal shift matters not for zero - any four bytes
could be given as the shift, ending in \x00, but 0 is the simplest.

--
Paul "LeoNerd" Evans

leonerd@leonerd= .org.uk
ICQ# 4135350 =A0 =A0 =A0 | =A0Registered Linux# 179460
http://www.leonerd= .org.uk/



--bcaec51a77fa39f12a04f33a89b1--