cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sammy Yu (JIRA)" <>
Subject [jira] Issue Comment Edited: (CASSANDRA-418) SSTable generation clash during compaction
Date Thu, 03 Sep 2009 02:58:33 GMT


Sammy Yu edited comment on CASSANDRA-418 at 9/2/09 7:58 PM:

Should we also change CFS.getTempSSTableFileName to just increment fileIndexGenerator_ once?

      was (Author: sammy.yu):
    Should we also change the getTempSSTableFileName to just increment fileIndexGenerator_

> SSTable generation clash during compaction
> ------------------------------------------
>                 Key: CASSANDRA-418
>                 URL:
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 0.4
>            Reporter: Sammy Yu
>            Assignee: Chris Goffinet
>             Fix For: 0.4
>         Attachments: 0001-CASSANDRA-418-Use-monotonically-increasing-generatio.patch
> We found that one of our node started getting timeouts for get_slice.  Looking further
we found that the CFS.ssTables_ references a SStable doesn't exist on the file system.
> Walking down the log we see that the sstable in question 6038 is being compacted onto
itself (in terms of filename file wise it is written to -tmp):
> system.log.2009-09-01: INFO [MINOR-COMPACTION-POOL:1] 2009-09-01 23:50:07,553
(line 1067) Compacting 
> [/mnt/var/cassandra/data/Digg/FriendActions-6037-Data.db,/mnt/var/cassandra/data/Digg/FriendActions-6038-Data.db,/mnt/var/cassandra/data/Digg/
> FriendActions-6040-Data.db,/mnt/var/cassandra/data/Digg/FriendActions-6042-Data.db]
> system.log.2009-09-01: INFO [MINOR-COMPACTION-POOL:1] 2009-09-01 23:51:43,727
(line 1209) Compacted to
> /mnt/var/cassandra/data/Digg/FriendActions-6038-Data.db.  0/1010269806 bytes for 9482/9373
keys read/written.  Time: 96173ms.
> It appears the generation number is generated by looking at the lowest number in the
list of files to be compacted and adding 1.  In this scenario it is 6037+1=6038.
> The code in CFS.doFileCompaction will remove the key and add the key back and remove
the key again, hence the error we were seeing.
> Should the generation number be generated via another way or should we update doFileCompaction
to be smarter?

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message