cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yuki Morishita (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-4205) SSTables are not updated with max timestamp on upgradesstables/compaction leading to non-optimal performance.
Date Thu, 03 May 2012 19:00:54 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-4205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13267690#comment-13267690
] 

Yuki Morishita commented on CASSANDRA-4205:
-------------------------------------------

Jonathan,

Re: patch, you have to bump up CURRENT_VERSION to "hd" but otherwise looking good to me.
                
> SSTables are not updated with max timestamp on upgradesstables/compaction leading to
non-optimal performance.
> -------------------------------------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-4205
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4205
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.0.0
>            Reporter: Thorkild Stray 
>            Assignee: Jonathan Ellis
>            Priority: Critical
>             Fix For: 1.0.10, 1.1.1
>
>         Attachments: 4205.txt
>
>
> We upgraded from 0.7.9 to 1.0.7 on a cluster with a heavy update load. After converting
all the reads to named column reads instead of get_slice calls, we noticed that we still weren't
getting the performance improvements implemented in CASSANDRA-2498. A single named column
read was still touching multiple SSTables according to nodetool cfhistograms. 
> To verify whether or not this was a reporting issue or a real issue, we ran multiple
tests with stress and noticed that it worked as expected. After changing stress so that it
ran the read/write test directly in the CF having issues (3 times stress & flush), we
noticed that stress also touched multiple SSTables (according to cfhistograms).
> So, the root of the problem is "something" left over from our pre-1.0 days. All SSTables
were upgraded with upgradesstables, and have been written and compacted many times since the
upgrade (4 months ago). The usage pattern for this CF is that it is constantly read and updated
(overwritten), but no deletes. 
> After discussing the problem with Brandon Williams on #cassandra, it seems the problem
might be because a max timestamp has never been written for the old SSTables that were upgraded
from pre 1.0. They have only been compacted, and the max timestamp is not recorded during
compactions. 
> A suggested fix is to special case this in upgradesstables so that a max timestamp always
exists for all SSTables. 
> {panel}
> 06:08 < driftx> thorkild_: tx.  The thing is we don't record the max timestamp
on compactions, but we can do it specially for upgradesstables.
> 06:08 < driftx> so, nothing in... nothing out.
> 06:10 < thorkild_> driftx: ah, so when you upgrade from before the metadata was
written, and that data is only feed through upgradesstables and compactions -> never properly
written?
> 06:10 < thorkild_> that makes sense.
> 06:11 < driftx> right, we never create it, we just reuse it :(
> {panel}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message