cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jeff Jirsa (JIRA)" <>
Subject [jira] [Updated] (CASSANDRA-9754) Make index info heap friendly for large CQL partitions
Date Tue, 25 Oct 2016 23:57:58 GMT


Jeff Jirsa updated CASSANDRA-9754:
    Attachment: 0f8e28c220fd5af6c7b5dd2d3dab6936c4aa4b6b.patch

For tables with many non-indexed rows, this patch (0f8e28c220fd5af6c7b5dd2d3dab6936c4aa4b6b.patch)
will pack non-indexed row entries behind the 4k aligned birch segments rather than aligning
to the boundary for each (nearly empty) entry. That is - rather than aligning AFTER each time
we write a new segment/subsegment, we align before we write a new birch tree/segment, and
for non-aligned segments/subsegments, we skip the align entirely to save space. 

I've not tested the performance impact of this patch, but in terms of size on disk, it creates
an -Index component approximately 98% smaller than the  birch/9754 patch for an example table
where the mean partition size is ~5k .  

Thoughts on this, [~mkjellman] ? 

> Make index info heap friendly for large CQL partitions
> ------------------------------------------------------
>                 Key: CASSANDRA-9754
>                 URL:
>             Project: Cassandra
>          Issue Type: Improvement
>            Reporter: sankalp kohli
>            Assignee: Michael Kjellman
>            Priority: Minor
>             Fix For: 4.x
>         Attachments: 0f8e28c220fd5af6c7b5dd2d3dab6936c4aa4b6b.patch, gc_collection_times_with_birch.png,
gc_collection_times_without_birch.png, gc_counts_with_birch.png, gc_counts_without_birch.png,
perf_cluster_1_with_birch_read_latency_and_counts.png, perf_cluster_1_with_birch_write_latency_and_counts.png,
perf_cluster_2_with_birch_read_latency_and_counts.png, perf_cluster_2_with_birch_write_latency_and_counts.png,
perf_cluster_3_without_birch_read_latency_and_counts.png, perf_cluster_3_without_birch_write_latency_and_counts.png
>  Looking at a heap dump of 2.0 cluster, I found that majority of the objects are IndexInfo
and its ByteBuffers. This is specially bad in endpoints with large CQL partitions. If a CQL
partition is say 6,4GB, it will have 100K IndexInfo objects and 200K ByteBuffers. This will
create a lot of churn for GC. Can this be improved by not creating so many objects?

This message was sent by Atlassian JIRA

View raw message