Mailing-List: contact dev-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cassandra.apache.org
Received-SPF: pass (athena.apache.org: domain of sylvain@datastax.com
 designates 209.85.161.172 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <1313106794.3184.5.camel@greenlantern.local>
References: 
 <CAO=WuYJXhJBuPmJ15rnU221E_15wWBXZD2ccA6e+ii2CV8na=g@mail.gmail.com>
 <CAKkz8Q35cHkwE_RLBFtcA7YwTQkKBf4+HL_vtAeCH=7UjkDXuw@mail.gmail.com>
 <1313021069.28827.17.camel@greenlantern.local>
 <1313030454.3184.2.camel@greenlantern.local>
 <CAKkz8Q1XVmmN-3pOJT7Tw9t-4LbVu080Q=dX0JiBpzcZh_hu=w@mail.gmail.com>
 <1313106794.3184.5.camel@greenlantern.local>
From: Sylvain Lebresne <sylvain@datastax.com>
Date: Mon, 29 Aug 2011 13:41:15 +0200
Message-ID: 
 <CAKkz8Q05HpSh58t9J6R6kV_b3-9bbyucdiuKaJJA2YPp42kuww@mail.gmail.com>
Subject: Re: Possible Bug in 0.8.2 with DynamicComposite range scans?
To: dev@cassandra.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

On Fri, Aug 12, 2011 at 1:53 AM, Todd Nine <todd@spidertracks.com> wrote:
> Hi Sylvain,
> =A0Could you share the command you used to create the CF with the
> pre-defined aliases? =A0I'm constantly receiving this error when I use th=
e
> following command.
>
> Syntax error at position 63: missing EOF at '('
>
>
> create column family test2 with
> comparator=3DDynamicCompositeType(a=3D>AsciiType,b=3D>BytesType,i=3D>Inte=
gerType,x=3D>LexicalUUIDType,l=3D>LongType,t=3D>TimeUUIDType,s=3D>UTF8Type,=
u=3D>UUIDType,A=3D>AsciiType(reversed=3Dtrue),B=3D>BytesType(reversed=3Dtru=
e),I=3D>IntegerType(reversed=3Dtrue),X=3D>LexicalUUIDType(reversed=3Dtrue),=
L=3D>LongType(reversed=3Dtrue),T=3D>TimeUUIDType(reversed=3Dtrue),S=3D>UTF8=
Type(reversed=3Dtrue),U=3D>UUIDType(reversed=3Dtrue)) =A0and key_validation=
_class=3DAsciiType and default_validation_class=3DAsciiType;
>
> I'll keep digging. =A0I have a it's most likely a disconnect between the
> serialization of the type aliases on the client and the treatment of
> aliases in Cassandra.

I think you're missing quotes. Try
comparator=3D'DynamicCompositeType(a=3D>AsciiType,b=3D>BytesType,...)'. Tha=
t
worked for me.

--
Sylvain

>
>
> Thanks,
> Todd
>
>
> On Thu, 2011-08-11 at 10:57 +0200, Sylvain Lebresne wrote:
>
>> I've tried using the longer values (converted as hex, so respectively
>> 0000012D3FDFA400 and 0000012D4E52B800, since that is what
>> the CLI can understand) and with the exact DynamicCompositeType
>> declaration above (almost exact, I've fixed DynamicComposite ->
>> DynamicCompositeType). I've tried with and without actually using
>> the aliases (that is, I tried with the aliases declared but without usin=
g
>> them in the values I sent). I always got the right. I really think this
>> is on the hector side at this point.
>>
>> >> When performing the range scan for my test the method
>> >> "getColumnComparator" on line 106 of the SliceQueryFilter is invoked.
>> >> It's using the BytesType comparator, so it is comparing the second
>> >> component.
>>
>> Well, maybe the problem is when you declare the column family then.
>> Are you set you correctly set the DynamicCompositeType when you
>> create the column family (check with the CLI maybe, using describe keysp=
ace).
>>
>> Because the getColumnComparator should return the DynamicCompositeType,
>> not BytesType. The comparison of the different component is done interna=
lly
>> in DynamicCompositeType.
>>
>> >> However, the "reversed" boolean flag is set to false, so it's not
>> >> correctly utilizing the columeReverseComparator instance when
>> >> performing range scans.
>>
>> This is right, because the slice query itself is not reversed. Again, th=
e
>> comparison is internal to DynamicCompositeType that will use the Reverse=
Type
>> comparator to compare the second component.
>>
>> --
>> Sylvain
>>
>> >> This seems to be a disconnect between when a column is specified as
>> >> "reversed" in the component itself, and reversed is specified in the
>> >> range query. =A0For each component, wouldn't you need to do this?
>> >>
>> >> reversed =3D user reversed ^ composite reversed
>> >>
>> >> This is the table I came up with for range scanning. =A0True is forwa=
rd,
>> >> false is reverse
>> >>
>> >> User =A0 =A0Component =A0 =A0Scan direction
>> >> false =A0 false =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 false
>> >> false =A0 true =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 tr=
ue
>> >> true =A0 =A0 false =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 tr=
ue
>> >> true =A0 =A0 true =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=
 false
>> >>
>> >>
>> >>
>> >>
>> >> Thanks,
>> >>
>> >>
>> >> --
>> >> todd
>> >> CHIEF SOFTWARE ENGINEER
>> >>
>> >> todd nine| spidertracks ltd | =A0117a the square
>> >> po box 5203 | palmerston north 4441 | new zealand
>> >> P: +64 6 353 3395
>> >> E: todd@spidertracks.co.nz W: www.spidertracks.com
>> >>
>> >>
>> >>
>> >> On Wed, 2011-08-10 at 12:26 +0200, Sylvain Lebresne wrote:
>> >> > Well, this seem to be on the hector side.
>> >> >
>> >> > I've tried the same example using the CLI, and:
>> >> >
>> >> > [default@unknown] create keyspace test;
>> >> > 642e6f90-c336-11e0-0000-242d50cf1fd5
>> >> > Waiting for schema agreement...
>> >> > ... schemas agree across the cluster
>> >> > [default@unknown] use test;
>> >> > Authenticated to keyspace: test
>> >> > [default@test] create column family foobar with
>> >> > comparator=3DDynamicCompositeType and key_validation_class=3DAsciiT=
ype and
>> >> > default_validation_class=3DAsciiType;
>> >> > 40032380-c337-11e0-0000-242d50cf1fd5
>> >> > Waiting for schema agreement...
>> >> > ... schemas agree across the cluster
>> >> > [default@test] set foobar[k]['UTF8Type@jeans:BytesType(reversed=3Dt=
rue)@1'] =3D a;
>> >> > Value inserted.
>> >> > [default@test] get foobar[k];
>> >> > =3D> (column=3DUTF8Type@jeans:BytesType(reversed=3Dtrue)@01, value=
=3Da,
>> >> > timestamp=3D1312970389512000)
>> >> > Returned 1 results.
>> >> > [default@test] set foobar[k]['UTF8Type@jeans:BytesType(reversed=3Dt=
rue)@2'] =3D a;
>> >> > Value inserted.
>> >> > [default@test] get foobar[k];
>> >> > =3D> (column=3DUTF8Type@jeans:BytesType(reversed=3Dtrue)@02, value=
=3Da,
>> >> > timestamp=3D1312970410712000)
>> >> > =3D> (column=3DUTF8Type@jeans:BytesType(reversed=3Dtrue)@01, value=
=3Da,
>> >> > timestamp=3D1312970389512000)
>> >> > Returned 2 results.
>> >> >
>> >> > Now, the last query is not exactly the one you do, since it does a =
full row
>> >> > query but the CLI don't support setting the start and end of a slic=
e. However,
>> >> > I have tried hard-coding the exact query into the CLI (with
>> >> > start=3D'UTF8Type@jeans'
>> >> > and end=3D'UTF8Type@jeans:!'), and it still returns the columns in =
the columns
>> >> > in the right order (with the biggest second component first).
>> >> >
>> >> > --
>> >> > Sylvain
>> >> >
>> >> > On Wed, Aug 10, 2011 at 9:26 AM, Todd Nine <todd@spidertracks.com> =
wrote:
>> >> > > Hi guys,
>> >> > > =A0 I've been dealing with a problem in my JPA plugin for a coupl=
e days
>> >> > > now. =A0I've been able to create a native test in 0.8.2 that repr=
oduces
>> >> > > the issue. =A0Here is the test.
>> >> > >
>> >> > >
>> >> > > https://gist.github.com/3ce70eab8102d2555626
>> >> > >
>> >> > >
>> >> > > Essentially, here is what is happening.
>> >> > >
>> >> > > A dynamic composite with the following ordering is created in a c=
olumn
>> >> > >
>> >> > > UTF8Type+BytesType(reversed=3D
>> >> > > true).
>> >> > >
>> >> > > 2 columns are then inserted, without composite encoding, these ar=
e the 2 values
>> >> > >
>> >> > > "jeans" + 1293840000000L
>> >> > >
>> >> > > "jeans" + 1294099200000L
>> >> > >
>> >> > >
>> >> > > Here are the byte values (with spaces added to make the encoding =
of
>> >> > > the composite easier to read) =A0The format is 4 byte comparator,=
 4 byte
>> >> > > length, n field bytes, 1 byte comparator, then repeats
>> >> > >
>> >> > > Inserted:
>> >> > >
>> >> > > 8073 0005 6a65616e73 00 =A0 =A08042 0008 0000012d4b889b80 00
>> >> > > 8073 0005 6a65616e73 00 =A0 =A08042 0008 0000012d3c158780 00
>> >> > >
>> >> > > Query start
>> >> > >
>> >> > > 8073 0005 6a65616e73 00
>> >> > >
>> >> > > Query end
>> >> > >
>> >> > > 8073 0005 6a65616e73 01
>> >> > >
>> >> > > Returned from Hector Results
>> >> > >
>> >> > > 8073 0005 6a65616e73 00 =A0 =A08042 0008 0000012d3c158780 00
>> >> > > 8073 0005 6a65616e73 00 =A0 =A08042 0008 0000012d4b889b80 00
>> >> > >
>> >> > >
>> >> > > Given that the first value is sorted normally, and the second val=
ue is
>> >> > > reversed, I would expect the higher long value to appear before t=
he
>> >> > > lower one (the longs are dates) when the first value in the compo=
site
>> >> > > is equal. =A0Is this the expected behavior, or is this a bug?
>> >> > >
>> >> > > Thanks,
>> >> > > Todd
>> >> > >
>> >
>> >
>