Mailing-List: contact dev-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cassandra.apache.org
Received-SPF: neutral (athena.apache.org: local policy)
Sender: Todd Nine <todd@spidertracks.co.nz>
Subject: Re: Possible Bug in 0.8.2 with DynamicComposite range scans?
From: Todd Nine <todd@spidertracks.com>
To: dev@cassandra.apache.org
In-Reply-To: 
 <CAKkz8Q35cHkwE_RLBFtcA7YwTQkKBf4+HL_vtAeCH=7UjkDXuw@mail.gmail.com>
References: 
 <CAO=WuYJXhJBuPmJ15rnU221E_15wWBXZD2ccA6e+ii2CV8na=g@mail.gmail.com>
	 <CAKkz8Q35cHkwE_RLBFtcA7YwTQkKBf4+HL_vtAeCH=7UjkDXuw@mail.gmail.com>
Content-Type: multipart/alternative; boundary="=-Rg2vJqEkfoP85VwnpDOd"
Date: Thu, 11 Aug 2011 12:04:29 +1200
Message-ID: <1313021069.28827.17.camel@greenlantern.local>
Mime-Version: 1.0

--=-Rg2vJqEkfoP85VwnpDOd
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit

Hi Sylvain,
  I did a bit more digging, and I may have found the issue, but I
haven't yet determined the root cause.   This is all from the 0.8.2
release source.

When performing the range scan for my test the method
"getColumnComparator" on line 106 of the SliceQueryFilter is invoked.
It's using the BytesType comparator, so it is comparing the second
component.  

However, the "reversed" boolean flag is set to false, so it's not
correctly utilizing the columeReverseComparator instance when performing
range scans. 

This seems to be a disconnect between when a column is specified as
"reversed" in the component itself, and reversed is specified in the
range query.  For each component, wouldn't you need to do this?

reversed = user reversed ^ composite reversed

This is the table I came up with for range scanning.  True is forward,
false is reverse

User    Component    Scan direction
false   false                         false
false   true                           true
true     false                         true
true     true                           false


Thanks,


-- 
todd 
CHIEF SOFTWARE ENGINEER

todd nine| spidertracks ltd |  117a the square 
po box 5203 | palmerston north 4441 | new zealand 
P: +64 6 353 3395
E: todd@spidertracks.co.nz W: www.spidertracks.com 


On Wed, 2011-08-10 at 12:26 +0200, Sylvain Lebresne wrote:

> Well, this seem to be on the hector side.
> 
> I've tried the same example using the CLI, and:
> 
> [default@unknown] create keyspace test;
> 642e6f90-c336-11e0-0000-242d50cf1fd5
> Waiting for schema agreement...
> ... schemas agree across the cluster
> [default@unknown] use test;
> Authenticated to keyspace: test
> [default@test] create column family foobar with
> comparator=DynamicCompositeType and key_validation_class=AsciiType and
> default_validation_class=AsciiType;
> 40032380-c337-11e0-0000-242d50cf1fd5
> Waiting for schema agreement...
> ... schemas agree across the cluster
> [default@test] set foobar[k]['UTF8Type@jeans:BytesType(reversed=true)@1'] = a;
> Value inserted.
> [default@test] get foobar[k];
> => (column=UTF8Type@jeans:BytesType(reversed=true)@01, value=a,
> timestamp=1312970389512000)
> Returned 1 results.
> [default@test] set foobar[k]['UTF8Type@jeans:BytesType(reversed=true)@2'] = a;
> Value inserted.
> [default@test] get foobar[k];
> => (column=UTF8Type@jeans:BytesType(reversed=true)@02, value=a,
> timestamp=1312970410712000)
> => (column=UTF8Type@jeans:BytesType(reversed=true)@01, value=a,
> timestamp=1312970389512000)
> Returned 2 results.
> 
> Now, the last query is not exactly the one you do, since it does a full row
> query but the CLI don't support setting the start and end of a slice. However,
> I have tried hard-coding the exact query into the CLI (with
> start='UTF8Type@jeans'
> and end='UTF8Type@jeans:!'), and it still returns the columns in the columns
> in the right order (with the biggest second component first).
> 
> --
> Sylvain
> 
> On Wed, Aug 10, 2011 at 9:26 AM, Todd Nine <todd@spidertracks.com> wrote:
> > Hi guys,
> >   I've been dealing with a problem in my JPA plugin for a couple days
> > now.  I've been able to create a native test in 0.8.2 that reproduces
> > the issue.  Here is the test.
> >
> >
> > https://gist.github.com/3ce70eab8102d2555626
> >
> >
> > Essentially, here is what is happening.
> >
> > A dynamic composite with the following ordering is created in a column
> >
> > UTF8Type+BytesType(reversed=
> > true).
> >
> > 2 columns are then inserted, without composite encoding, these are the 2 values
> >
> > "jeans" + 1293840000000L
> >
> > "jeans" + 1294099200000L
> >
> >
> > Here are the byte values (with spaces added to make the encoding of
> > the composite easier to read)  The format is 4 byte comparator, 4 byte
> > length, n field bytes, 1 byte comparator, then repeats
> >
> > Inserted:
> >
> > 8073 0005 6a65616e73 00    8042 0008 0000012d4b889b80 00
> > 8073 0005 6a65616e73 00    8042 0008 0000012d3c158780 00
> >
> > Query start
> >
> > 8073 0005 6a65616e73 00
> >
> > Query end
> >
> > 8073 0005 6a65616e73 01
> >
> > Returned from Hector Results
> >
> > 8073 0005 6a65616e73 00    8042 0008 0000012d3c158780 00
> > 8073 0005 6a65616e73 00    8042 0008 0000012d4b889b80 00
> >
> >
> > Given that the first value is sorted normally, and the second value is
> > reversed, I would expect the higher long value to appear before the
> > lower one (the longs are dates) when the first value in the composite
> > is equal.  Is this the expected behavior, or is this a bug?
> >
> > Thanks,
> > Todd
> >

--=-Rg2vJqEkfoP85VwnpDOd--