cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Viktor Jevdokimov <Viktor.Jevdoki...@adform.com>
Subject RE: tombstones problem with 1.0.8
Date Fri, 23 Mar 2012 08:04:18 GMT
Yes, continued deletions of the same columns/rows will prevent removing them from final sstable
upon compaction due to new timestamp.
You're getting sliding tombstone gc grace period in that case.

During compaction of selected sstables Cassandra checks the whole Column Family for the latest
timestamp of the column/row, including other sstables and memtable.

You need to review your application logic.




Best regards/ Pagarbiai



Viktor Jevdokimov

Senior Developer



Email:  Viktor.Jevdokimov@adform.com

Phone: +370 5 212 3063. Fax: +370 5 261 0453

J. Jasinskio 16C, LT-01112 Vilnius, Lithuania






[Adform news]<http://www.adform.com/>

[Visit us!]

Follow:


[twitter]<http://twitter.com/#!/adforminsider>

Visit our blog<http://www.adform.com/site/blog>



Disclaimer: The information contained in this message and attachments is intended solely for
the attention and use of the named addressee and may be confidential. If you are not the intended
recipient, you are reminded that the information remains the property of the sender. You must
not use, disclose, distribute, copy, print or rely on this e-mail. If you have received this
message in error, please contact the sender immediately and irrevocably delete this message
and any copies.

From: Ross Black [mailto:ross.w.black@gmail.com]
Sent: Friday, March 23, 2012 07:16
To: user@cassandra.apache.org
Subject: Re: tombstones problem with 1.0.8

Hi Victor,

Thanks for your response.

Is there a possibility that continual deletions during compact could be blocking removal of
the tombstones?  The full manual compact takes about 4 hours per node for our data, so there
is a large number of deletes occurring during that time.

This is the description from cassandra-cli

  Replication Strategy: org.apache.cassandra.locator.SimpleStrategy
  Durable Writes: true
    Options: [replication_factor:3]
  Column Families:
    ColumnFamily: weekly
      Key Validation Class: org.apache.cassandra.db.marshal.BytesType
      Default column value validator: org.apache.cassandra.db.marshal.BytesType
      Columns sorted by: org.apache.cassandra.db.marshal.BytesType
      Row cache size / save period in seconds / keys to save : 0.0/0/all
      Row Cache Provider: org.apache.cassandra.cache.SerializingCacheProvider
      Key cache size / save period in seconds: 200000.0/14400
      GC grace seconds: 3600
      Compaction min/max thresholds: 3/8
      Read repair chance: 1.0
      Replicate on write: true
      Bloom Filter FP chance: default
      Built indexes: []
      Compaction Strategy: org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy


Ross


On 23 March 2012 02:55, Viktor Jevdokimov <Viktor.Jevdokimov@adform.com<mailto:Viktor.Jevdokimov@adform.com>>
wrote:
Just tested 1.0.8 before upgrading from 1.0.7: tombstones created by TTL or by delete operation
are perfectly deleted after either compaction or cleanup.
Have no idea about any other settings than gc_grace_seconds, check you schema from cassandra-cli.






Best regards/ Pagarbiai



Viktor Jevdokimov

Senior Developer



Email:  Viktor.Jevdokimov@adform.com<mailto:Viktor.Jevdokimov@adform.com>

Phone: +370 5 212 3063<tel:%2B370%205%20212%203063>. Fax: +370 5 261 0453<tel:%2B370%205%20261%200453>

J. Jasinskio 16C, LT-01112 Vilnius, Lithuania






[Adform news]<http://www.adform.com/>

[Visit us!]

Follow:



[twitter]<http://twitter.com/#%21/adforminsider>


Visit our blog<http://www.adform.com/site/blog>


Disclaimer: The information contained in this message and attachments is intended solely for
the attention and use of the named addressee and may be confidential. If you are not the intended
recipient, you are reminded that the information remains the property of the sender. You must
not use, disclose, distribute, copy, print or rely on this e-mail. If you have received this
message in error, please contact the sender immediately and irrevocably delete this message
and any copies.

From: Ross Black [mailto:ross.w.black@gmail.com<mailto:ross.w.black@gmail.com>]
Sent: Thursday, March 22, 2012 03:38
To: user@cassandra.apache.org<mailto:user@cassandra.apache.org>
Subject: tombstones problem with 1.0.8

Hi,

We recently moved from 0.8.2 to 1.0.8 and the behaviour seems to have changed so that tombstones
are now not being deleted.

Our application continually adds and removes columns from Cassandra.  We have set a short
gc_grace time (3600) since our application would automatically delete zombies if they appear.
Under 0.8.2, the tombstones remained at a relatively constant number.
Under 1.0.8, the tombstones have been continually increasing so that they exceed the size
of our real data (at this stage we have over 100G of tombstones).
Even after running a full compact the new compacted SSTable contains a massive number of tombstones,
many that are several weeks old.

Have I missed some new configuration option to allow deletion of tombstones?

I also noticed that one of the changes between 0.8.2 and 1.0.8 was https://issues.apache.org/jira/browse/CASSANDRA-2786
which changed code to "avoid dropping tombstones when they might still be needed to shadow
data in another sstable".
Could this be having an impact since we continually add and remove columns even while a major
compact is executing?


Thanks,
Ross


Mime
View raw message