incubator-cassandra-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Todd Nine <t...@spidertracks.com>
Subject Re: Possible Bug in 0.8.2 with DynamicComposite range scans?
Date Thu, 11 Aug 2011 23:53:14 GMT
Hi Sylvain,
  Could you share the command you used to create the CF with the
pre-defined aliases?  I'm constantly receiving this error when I use the
following command.

Syntax error at position 63: missing EOF at '('


create column family test2 with
comparator=DynamicCompositeType(a=>AsciiType,b=>BytesType,i=>IntegerType,x=>LexicalUUIDType,l=>LongType,t=>TimeUUIDType,s=>UTF8Type,u=>UUIDType,A=>AsciiType(reversed=true),B=>BytesType(reversed=true),I=>IntegerType(reversed=true),X=>LexicalUUIDType(reversed=true),L=>LongType(reversed=true),T=>TimeUUIDType(reversed=true),S=>UTF8Type(reversed=true),U=>UUIDType(reversed=true))
 and key_validation_class=AsciiType and default_validation_class=AsciiType;

I'll keep digging.  I have a it's most likely a disconnect between the
serialization of the type aliases on the client and the treatment of
aliases in Cassandra.


Thanks,
Todd


On Thu, 2011-08-11 at 10:57 +0200, Sylvain Lebresne wrote:

> I've tried using the longer values (converted as hex, so respectively
> 0000012D3FDFA400 and 0000012D4E52B800, since that is what
> the CLI can understand) and with the exact DynamicCompositeType
> declaration above (almost exact, I've fixed DynamicComposite ->
> DynamicCompositeType). I've tried with and without actually using
> the aliases (that is, I tried with the aliases declared but without using
> them in the values I sent). I always got the right. I really think this
> is on the hector side at this point.
> 
> >> When performing the range scan for my test the method
> >> "getColumnComparator" on line 106 of the SliceQueryFilter is invoked.
> >> It's using the BytesType comparator, so it is comparing the second
> >> component.
> 
> Well, maybe the problem is when you declare the column family then.
> Are you set you correctly set the DynamicCompositeType when you
> create the column family (check with the CLI maybe, using describe keyspace).
> 
> Because the getColumnComparator should return the DynamicCompositeType,
> not BytesType. The comparison of the different component is done internally
> in DynamicCompositeType.
> 
> >> However, the "reversed" boolean flag is set to false, so it's not
> >> correctly utilizing the columeReverseComparator instance when
> >> performing range scans.
> 
> This is right, because the slice query itself is not reversed. Again, the
> comparison is internal to DynamicCompositeType that will use the ReverseType
> comparator to compare the second component.
> 
> --
> Sylvain
> 
> >> This seems to be a disconnect between when a column is specified as
> >> "reversed" in the component itself, and reversed is specified in the
> >> range query.  For each component, wouldn't you need to do this?
> >>
> >> reversed = user reversed ^ composite reversed
> >>
> >> This is the table I came up with for range scanning.  True is forward,
> >> false is reverse
> >>
> >> User    Component    Scan direction
> >> false   false                         false
> >> false   true                           true
> >> true     false                         true
> >> true     true                           false
> >>
> >>
> >>
> >>
> >> Thanks,
> >>
> >>
> >> --
> >> todd
> >> CHIEF SOFTWARE ENGINEER
> >>
> >> todd nine| spidertracks ltd |  117a the square
> >> po box 5203 | palmerston north 4441 | new zealand
> >> P: +64 6 353 3395
> >> E: todd@spidertracks.co.nz W: www.spidertracks.com
> >>
> >>
> >>
> >> On Wed, 2011-08-10 at 12:26 +0200, Sylvain Lebresne wrote:
> >> > Well, this seem to be on the hector side.
> >> >
> >> > I've tried the same example using the CLI, and:
> >> >
> >> > [default@unknown] create keyspace test;
> >> > 642e6f90-c336-11e0-0000-242d50cf1fd5
> >> > Waiting for schema agreement...
> >> > ... schemas agree across the cluster
> >> > [default@unknown] use test;
> >> > Authenticated to keyspace: test
> >> > [default@test] create column family foobar with
> >> > comparator=DynamicCompositeType and key_validation_class=AsciiType and
> >> > default_validation_class=AsciiType;
> >> > 40032380-c337-11e0-0000-242d50cf1fd5
> >> > Waiting for schema agreement...
> >> > ... schemas agree across the cluster
> >> > [default@test] set foobar[k]['UTF8Type@jeans:BytesType(reversed=true)@1']
= a;
> >> > Value inserted.
> >> > [default@test] get foobar[k];
> >> > => (column=UTF8Type@jeans:BytesType(reversed=true)@01, value=a,
> >> > timestamp=1312970389512000)
> >> > Returned 1 results.
> >> > [default@test] set foobar[k]['UTF8Type@jeans:BytesType(reversed=true)@2']
= a;
> >> > Value inserted.
> >> > [default@test] get foobar[k];
> >> > => (column=UTF8Type@jeans:BytesType(reversed=true)@02, value=a,
> >> > timestamp=1312970410712000)
> >> > => (column=UTF8Type@jeans:BytesType(reversed=true)@01, value=a,
> >> > timestamp=1312970389512000)
> >> > Returned 2 results.
> >> >
> >> > Now, the last query is not exactly the one you do, since it does a full
row
> >> > query but the CLI don't support setting the start and end of a slice. However,
> >> > I have tried hard-coding the exact query into the CLI (with
> >> > start='UTF8Type@jeans'
> >> > and end='UTF8Type@jeans:!'), and it still returns the columns in the columns
> >> > in the right order (with the biggest second component first).
> >> >
> >> > --
> >> > Sylvain
> >> >
> >> > On Wed, Aug 10, 2011 at 9:26 AM, Todd Nine <todd@spidertracks.com>
wrote:
> >> > > Hi guys,
> >> > >   I've been dealing with a problem in my JPA plugin for a couple days
> >> > > now.  I've been able to create a native test in 0.8.2 that reproduces
> >> > > the issue.  Here is the test.
> >> > >
> >> > >
> >> > > https://gist.github.com/3ce70eab8102d2555626
> >> > >
> >> > >
> >> > > Essentially, here is what is happening.
> >> > >
> >> > > A dynamic composite with the following ordering is created in a column
> >> > >
> >> > > UTF8Type+BytesType(reversed=
> >> > > true).
> >> > >
> >> > > 2 columns are then inserted, without composite encoding, these are
the 2 values
> >> > >
> >> > > "jeans" + 1293840000000L
> >> > >
> >> > > "jeans" + 1294099200000L
> >> > >
> >> > >
> >> > > Here are the byte values (with spaces added to make the encoding of
> >> > > the composite easier to read)  The format is 4 byte comparator, 4
byte
> >> > > length, n field bytes, 1 byte comparator, then repeats
> >> > >
> >> > > Inserted:
> >> > >
> >> > > 8073 0005 6a65616e73 00    8042 0008 0000012d4b889b80 00
> >> > > 8073 0005 6a65616e73 00    8042 0008 0000012d3c158780 00
> >> > >
> >> > > Query start
> >> > >
> >> > > 8073 0005 6a65616e73 00
> >> > >
> >> > > Query end
> >> > >
> >> > > 8073 0005 6a65616e73 01
> >> > >
> >> > > Returned from Hector Results
> >> > >
> >> > > 8073 0005 6a65616e73 00    8042 0008 0000012d3c158780 00
> >> > > 8073 0005 6a65616e73 00    8042 0008 0000012d4b889b80 00
> >> > >
> >> > >
> >> > > Given that the first value is sorted normally, and the second value
is
> >> > > reversed, I would expect the higher long value to appear before the
> >> > > lower one (the longs are dates) when the first value in the composite
> >> > > is equal.  Is this the expected behavior, or is this a bug?
> >> > >
> >> > > Thanks,
> >> > > Todd
> >> > >
> >
> >

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message