cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ilya Maykov <ivmay...@gmail.com>
Subject Re: Seeing very weird results on 0.6.2 when paginating through a ColumnFamily with get_slice()
Date Fri, 16 Jul 2010 03:42:13 GMT
The column names are arbitrary strings, so it's not obvious what the
"next" value should be at any step. So, I just set the start of the
next page to the end of the last page and eliminate the duplicate
value when joining the 2 pages together.

The paging direction does not matter in my case, as I just want to get
the entire column family. And in the vast majority of cases (probably
99%+) the paginated get method returns the correct results. But
sometimes it does not.

-- Ilya



On Thu, Jul 15, 2010 at 8:35 PM, Paul Brown <paulrbrown@gmail.com> wrote:
>
> You should make sure that your directions and interval endpoints are chosen correctly.
 I recall the semantics of the call being like an old-school for with the descending flag
as a step of +1 or -1.
>
> --
> Spelling by mobile.
>
> On Jul 15, 2010, at 20:19, Ilya Maykov <ivmaykov@gmail.com> wrote:
>
>> Hi all,
>>
>> I'm trying to debug some pretty weird behavior when paginating through
>> a ColumnFamily with get_slice(). It basically looks like Cassandra
>> does not respect the limit parameter in the SlicePredicate, sometimes
>> returning more than limit columns. It also sometimes silently drops
>> columns. I'm reading using QUORUM, and all data was written using
>> QUORUM as well. The client is in Ruby.
>>
>> I am seeing basically non-deterministic behavior when paginating
>> through a column family (which is not being written concurrently).
>> Here is example output from my pagination method (code below) with
>> some extra debug prints added:
>>
>> irb(main):005:0> blah = get_entire_column_family(@cassandra,
>> "some_key", "some_cf", 100)
>> get_entire_column_family(@cass, "some_key", "some_cf", 100) ...
>> 100/6648 ... 199/6648 ... 354/6648 ... 453/6648 ... 552/6648 ...
>> 689/6648 ... 788/6648 ... 887/6648 ... 1048/6648 ... 1147/6648 ...
>> 1246/6648 ... 1377/6648 ... 1476/6648 ... 1575/6648 ... 1674/6648 ...
>> 1773/6648 ... 1908/6648 ... 2051/6648 ... 2150/6648 ... 2249/6648 ...
>> 2348/6648 ... <snip> ... 6127/6648 ... 6127/6648 ... 6127/6648 ...
>> 6127/6648 ... 6127/6648 ...
>>
>> The N/6648 is just printing the retrieved columns / total columns at
>> each step of the pagination loop. It should be going up by 99 on each
>> iteration after the first (because the start of the next slice == the
>> last value in the current slice). But sometimes it jumps, indicating
>> that more than 100 values were returned from a single get_slice() call
>> (i.e. ... 199/6648 ... 354/6648 ...). And when it gets to the end of
>> the column family, we end up with fewer than 6648 columns on the
>> client side and the code gets stuck in an infinite loop.
>>
>> I've tried this several times from an interactive Ruby session and got
>> a different number of columns each time:
>> 6536/6648
>> 6127/6648
>> 6514/6648
>> However, once I set the limit to be > num_columns and read the entire
>> row as a single page, everything worked. And follow-up paginated reads
>> also return the entire row successfully. Not sure if that's because
>> the entire row is now in cache, or because something was wrong and
>> read-repair has fixed it. But since all of our reads and writes are
>> done using QUORUM, read-repair shouldn't matter, right?
>>
>> Here is the pagination code:
>>
>>    def get_entire_column_family(cassandra, row_key, column_family,
>> limit_per_slice)
>>      column_parent = CassandraThrift::ColumnParent.new(:column_family
>> => column_family, :super_column => nil)
>>      num_columns = cassandra.get_count(@keyspace, row_key,
>> column_parent, CassandraThrift::ConsistencyLevel::QUORUM)
>>      predicate = CassandraThrift::SlicePredicate.new
>>      predicate.slice_range = CassandraThrift::SliceRange.new(:start
>> => "", :finish => "", :reversed => false, :count => limit_per_slice)
>>      slice = cassandra.get_slice(@keyspace, row_key, column_parent,
>> predicate, CassandraThrift::ConsistencyLevel::QUORUM)
>>      result = slice
>>      while result.size < num_columns
>>        predicate = CassandraThrift::SlicePredicate.new
>>        predicate.slice_range = CassandraThrift::SliceRange.new(:start
>> => result.last.column.name,
>>            :finish => "", :reversed => false, :count => limit_per_slice)
>>        slice = cassandra.get_slice(@keyspace, row_key, column_parent,
>> predicate, CassandraThrift::ConsistencyLevel::QUORUM)
>>        # Because the start parameter to get_slice() is inclusive, we
>> should already have the first column of the
>>        # new slice in our result. We don't want to have 2 copies of
>> it, so drop it from the slice before concatenating.
>>        unless slice.nil? || slice.empty?
>>          if result.last.column.name == slice.first.column.name
>>            result.concat slice[1 .. slice.size-1]
>>          else
>>            result.concat slice
>>          end
>>        end
>>      end  # while
>>      return result
>>    end
>>
>> I guess I have several questions:
>> 1) Is this the proper way to paginate through a large column family
>> for a single row key? If not, what is the proper way? Some of our rows
>> are very big (hundreds of thousands of columns in the worst case), and
>> pagination is a must.
>> 2) Could this behavior be expected under some conditions (i.e. maybe
>> presence of tombstones or hints from when a node was down or other
>> weirdness?)
>> 2) Is this a known bug? (maybe related to
>> https://issues.apache.org/jira/browse/CASSANDRA-1145 and/or
>> https://issues.apache.org/jira/browse/CASSANDRA-1042 ?)
>> 3) If this is not a known bug, how should I proceed with investigating it?
>>
>> Thanks,
>>
>> -- Ilya
>

Mime
View raw message