cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Christian Spriegel (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-4153) Optimize truncate when snapshots are disabled or keyspace not durable
Date Sun, 15 Apr 2012 11:21:16 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-4153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254305#comment-13254305
] 

Christian Spriegel commented on CASSANDRA-4153:
-----------------------------------------------

Yes, you are right. That is why I call renewMemtable() instead. It drops the old memtable
and creates a new one:
{code}
        if (DatabaseDescriptor.isAutoSnapshot())
        {
            forceBlockingFlush(); // this was the old flush
        }
        else
        {
            Table.switchLock.writeLock().lock();
            try
            {
                for (ColumnFamilyStore cfs : concatWithIndexes())
                {
                    Memtable mt = cfs.getMemtableThreadSafe();
                    if (!mt.isClean() && !mt.isFrozen())
                    {
                        mt.cfs.data.renewMemtable(); // just drop the memtable
                    }
                }
            }
            finally
            {
                Table.switchLock.writeLock().unlock();
            }
        }
{code}
This code is for flushing the memtable that shall be truncated.

Unfortunetaly that is not all. In order to be able to delete the commitlog, truncate does
also flush all other memtables (Which probably has the worst impact on my testperformance).
These flushes however become obsolete if the CF does not use the commitlog (the keyspace that
the CF is in, to be more precise):
{code}
        KSMetaData ksm = Schema.instance.getKSMetaData(this.table.name);
        if(ksm.durableWrites)
        {
            CommitLog.instance.forceNewSegment();
            ReplayPosition position = CommitLog.instance.getContext();
            // now flush everyone else.  re-flushing ourselves is not necessary, but harmless
            for (ColumnFamilyStore cfs : ColumnFamilyStore.all())
                cfs.forceFlush(); // these flushes are obsolete if durableWrites are off
            waitForActiveFlushes();
            // if everything was clean, flush won't have called discard
            CommitLog.instance.discardCompletedSegments(metadata.cfId, position);
        }
{code}

btw: I ran my testsuite with the patched Cassandra and it did truncate properly. So the very
basic stuff should work, but I am not that sure about side effects :-)

Whilst we're at it, I have other questions:
# Do you I need to call Table.switchLock.writeLock().lock() for renewMemtable()?
# Are you ok with my sleep change? I think waiting a full 100ms is not neccessary, we just
want to ensure that currentTimeInMillis() advances.
                
> Optimize truncate when snapshots are disabled or keyspace not durable
> ---------------------------------------------------------------------
>
>                 Key: CASSANDRA-4153
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4153
>             Project: Cassandra
>          Issue Type: Improvement
>            Reporter: Christian Spriegel
>            Priority: Minor
>         Attachments: OptimizeTruncate_v1.diff
>
>
> My goal is to make truncate to be less IO intensive so that my junit tests run faster
(as already explained in CASSANDRA-3710). I think I have now a solution which does not change
too much:
> I created a patch that optimizes three things within truncate:
> - Skip the whole Commitlog.forceNewSegment/discardCompletedSegments, if durable_writes
are disabled for the keyspace.
> - With CASSANDRA-3710 implemented, truncate does not need to flush memtables to disk
when snapshots are disabled.
> - Reduce the sleep interval
> The patch works nicely for me. Applying it and disabling durable_writes/autoSnapshot
increased the speed of my testsuite vastly. I hope I did not overlook something.
> Let me know if my patch needs cleanup. I'd be glad to change it, if it means the patch
will get accepted.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message