incubator-cassandra-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Todd Nine <t...@spidertracks.com>
Subject Re: Possible Bug in 0.8.2 with DynamicComposite range scans?
Date Thu, 11 Aug 2011 02:40:54 GMT
Correction to my truth table.  True is reversed, false is forward.  I
just caught that typo.


User    Component    Scan direction
false   false          false
false   true           true
true    false          true
true    true           false


-- 
todd 
CHIEF SOFTWARE ENGINEER

todd nine| spidertracks ltd |  117a the square 
po box 5203 | palmerston north 4441 | new zealand 
P: +64 6 353 3395
E: todd@spidertracks.co.nz W: www.spidertracks.com 


On Thu, 2011-08-11 at 12:04 +1200, Todd Nine wrote:
> Hi Sylvain,
>   I did a bit more digging, and I may have found the issue, but I
> haven't yet determined the root cause.   This is all from the 0.8.2
> release source.
> 
> When performing the range scan for my test the method
> "getColumnComparator" on line 106 of the SliceQueryFilter is invoked.
> It's using the BytesType comparator, so it is comparing the second
> component.  
> 
> However, the "reversed" boolean flag is set to false, so it's not
> correctly utilizing the columeReverseComparator instance when
> performing range scans. 
> 
> This seems to be a disconnect between when a column is specified as
> "reversed" in the component itself, and reversed is specified in the
> range query.  For each component, wouldn't you need to do this?
> 
> reversed = user reversed ^ composite reversed
> 
> This is the table I came up with for range scanning.  True is forward,
> false is reverse
> 
> User    Component    Scan direction
> false   false                         false
> false   true                           true
> true     false                         true
> true     true                           false
> 
> 
> 
> 
> Thanks,
> 
> 
> -- 
> todd 
> CHIEF SOFTWARE ENGINEER
> 
> todd nine| spidertracks ltd |  117a the square 
> po box 5203 | palmerston north 4441 | new zealand 
> P: +64 6 353 3395
> E: todd@spidertracks.co.nz W: www.spidertracks.com 
> 
> 
> 
> On Wed, 2011-08-10 at 12:26 +0200, Sylvain Lebresne wrote: 
> > Well, this seem to be on the hector side.
> > 
> > I've tried the same example using the CLI, and:
> > 
> > [default@unknown] create keyspace test;
> > 642e6f90-c336-11e0-0000-242d50cf1fd5
> > Waiting for schema agreement...
> > ... schemas agree across the cluster
> > [default@unknown] use test;
> > Authenticated to keyspace: test
> > [default@test] create column family foobar with
> > comparator=DynamicCompositeType and key_validation_class=AsciiType and
> > default_validation_class=AsciiType;
> > 40032380-c337-11e0-0000-242d50cf1fd5
> > Waiting for schema agreement...
> > ... schemas agree across the cluster
> > [default@test] set foobar[k]['UTF8Type@jeans:BytesType(reversed=true)@1'] = a;
> > Value inserted.
> > [default@test] get foobar[k];
> > => (column=UTF8Type@jeans:BytesType(reversed=true)@01, value=a,
> > timestamp=1312970389512000)
> > Returned 1 results.
> > [default@test] set foobar[k]['UTF8Type@jeans:BytesType(reversed=true)@2'] = a;
> > Value inserted.
> > [default@test] get foobar[k];
> > => (column=UTF8Type@jeans:BytesType(reversed=true)@02, value=a,
> > timestamp=1312970410712000)
> > => (column=UTF8Type@jeans:BytesType(reversed=true)@01, value=a,
> > timestamp=1312970389512000)
> > Returned 2 results.
> > 
> > Now, the last query is not exactly the one you do, since it does a full row
> > query but the CLI don't support setting the start and end of a slice. However,
> > I have tried hard-coding the exact query into the CLI (with
> > start='UTF8Type@jeans'
> > and end='UTF8Type@jeans:!'), and it still returns the columns in the columns
> > in the right order (with the biggest second component first).
> > 
> > --
> > Sylvain
> > 
> > On Wed, Aug 10, 2011 at 9:26 AM, Todd Nine <todd@spidertracks.com> wrote:
> > > Hi guys,
> > >   I've been dealing with a problem in my JPA plugin for a couple days
> > > now.  I've been able to create a native test in 0.8.2 that reproduces
> > > the issue.  Here is the test.
> > >
> > >
> > > https://gist.github.com/3ce70eab8102d2555626
> > >
> > >
> > > Essentially, here is what is happening.
> > >
> > > A dynamic composite with the following ordering is created in a column
> > >
> > > UTF8Type+BytesType(reversed=
> > > true).
> > >
> > > 2 columns are then inserted, without composite encoding, these are the 2 values
> > >
> > > "jeans" + 1293840000000L
> > >
> > > "jeans" + 1294099200000L
> > >
> > >
> > > Here are the byte values (with spaces added to make the encoding of
> > > the composite easier to read)  The format is 4 byte comparator, 4 byte
> > > length, n field bytes, 1 byte comparator, then repeats
> > >
> > > Inserted:
> > >
> > > 8073 0005 6a65616e73 00    8042 0008 0000012d4b889b80 00
> > > 8073 0005 6a65616e73 00    8042 0008 0000012d3c158780 00
> > >
> > > Query start
> > >
> > > 8073 0005 6a65616e73 00
> > >
> > > Query end
> > >
> > > 8073 0005 6a65616e73 01
> > >
> > > Returned from Hector Results
> > >
> > > 8073 0005 6a65616e73 00    8042 0008 0000012d3c158780 00
> > > 8073 0005 6a65616e73 00    8042 0008 0000012d4b889b80 00
> > >
> > >
> > > Given that the first value is sorted normally, and the second value is
> > > reversed, I would expect the higher long value to appear before the
> > > lower one (the longs are dates) when the first value in the composite
> > > is equal.  Is this the expected behavior, or is this a bug?
> > >
> > > Thanks,
> > > Todd
> > >


Mime
View raw message