incubator-cassandra-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan Ellis <jbel...@gmail.com>
Subject Re: performance with a "large" number of supercolumns/columns
Date Thu, 08 Jul 2010 15:15:10 GMT
On Thu, Jul 8, 2010 at 12:18 AM, Terje Marthinussen
<tmarthinussen@gmail.com> wrote:
> 1.  There is just too much code and too many "layers" involved when hasNext
> is called. I suspect this requires a re-design and the
> google/common-collections may have to be thrown out. This would seem to be a
> pretty critical area of Cassandra code for performance, so I suspect the
> cost of making  functionality specifically tuned for Cassandra should be
> worth it.

That might be interesting.  I've looked at the apache collated
iterator code [the main thing we use commons-collections for] and it
looks pretty sloppy.  Google collections (guava in trunk) otoh is
probably better than what I could come up with.

Kevin from guava suggests they will add support for collated iteration
"eventually."  See
http://code.google.com/p/guava-libraries/issues/detail?id=119 and the
linked 220.

> 2. From what I understand, Columns from all SS/MemTables are merged before
> considering tombstones and applying the slice predicates.

Partly -- for slicing, we deserialize an index block at a time (from
each sstable) until the the slice predicate is satisfied (getNextBlock
in ColumnGroupReader).  For reversed queries this is unavoidable, but
it could be optimized better for forward queries.

For column name predicates we deserialize exactly the requested columns.

It should be possible to optimize both better wrt tombstones (instead
of throwing out "uninteresting" columns in a final pass, pass in the
deletion time they have to "beat" to be relevant to the
IColumnIterators).

> While this may make things somewhat hairy codewise, JVM performance/memory
> consumption tend to not be happy in scenarios with heavy object creation,
> and we get a lot of objects here.

Agreed.

> 3. It would seem like there might be good possibilities to do a fair bit of
> optimization for the case where a complete set of columns are fetched
> (slices with null for start/end predicates), but I am not currently sure how
> much time it would save and if it would be worth the work and added
> complexity to the

As I commented on
https://issues.apache.org/jira/browse/CASSANDRA-1207, one piece of
low-hanging fruit would be to allow skipping either the column index
or bloom filter for name-based or slice-based queries, respectively.

> I may make an attempt at 1. but I don't feel entirely like I understand
> enough of the Cassandra code yet to do any of the above at the moment so it
> may take a little while.

Feel free to catch us on IRC in #cassandra-dev if you want quicker answers. :)

-- 
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of Riptano, the source for professional Cassandra support
http://riptano.com

Mime
View raw message