cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Christian Esken (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (CASSANDRA-13005) Cassandra TWCS is not removing fully expired tables
Date Tue, 06 Dec 2016 13:55:58 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-13005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15725518#comment-15725518
] 

Christian Esken edited comment on CASSANDRA-13005 at 12/6/16 1:54 PM:
----------------------------------------------------------------------

I changed the sstabledump to list the metadata types, and it looks like only STATS are there,
and not even the HEADER:
{code}
# tools/bin/sstabledump mc-53346-big-Data.db 
sstableMetadata Map: {STATS=org.apache.cassandra.io.sstable.metadata.StatsMetadata@ea892389}
{code}

{code}
    public static CFMetaData metadataFromSSTable(Descriptor desc) throws IOException
    {
        if (!desc.version.storeRows())
            throw new IOException("pre-3.0 SSTable is not supported.");

       // EDIT: cesken Added  all metadata types for analyzing what is availabe in the SSTable
        EnumSet<MetadataType> types = EnumSet.of(MetadataType.STATS, MetadataType.HEADER,
MetadataType.VALIDATION, MetadataType.COMPACTION);
        Map<MetadataType, MetadataComponent> sstableMetadata = desc.getMetadataSerializer().deserialize(desc,
types);

        // EDIT: cesken Print the metadata types we found
        System.out.println("sstableMetadata Map: " + sstableMetadata);

        SerializationHeader.Component header = (SerializationHeader.Component) sstableMetadata.get(MetadataType.HEADER);
        // Hint: header == null here

        // Hint: Method call below crashes, as there is no   MetadataType.VALIDATION, so validationMetadata
== null
        IPartitioner partitioner = FBUtilities.newPartitioner(desc);

{code}



was (Author: cesken):
I changed the sstabledump to list the metadata types, and it looks like only STATS are there,
and not even the HEADER:
{code}
# tools/bin/sstabledump mc-53346-big-Data.db 
sstableMetadata Map: {STATS=org.apache.cassandra.io.sstable.metadata.StatsMetadata@ea892389}
{code}

{code}
    public static CFMetaData metadataFromSSTable(Descriptor desc) throws IOException
    {
        if (!desc.version.storeRows())
            throw new IOException("pre-3.0 SSTable is not supported.");

       // EDIT: cesken Added  all metadata types for analyzing what is availabe in the SSTable
        EnumSet<MetadataType> types = EnumSet.of(MetadataType.STATS, MetadataType.HEADER,
MetadataType.VALIDATION, MetadataType.COMPACTION);
        Map<MetadataType, MetadataComponent> sstableMetadata = desc.getMetadataSerializer().deserialize(desc,
types);

        // EDIT: cesken Print the metadata types we found
        System.out.println("sstableMetadata Map: " + sstableMetadata);

        SerializationHeader.Component header = (SerializationHeader.Component) sstableMetadata.get(MetadataType.HEADER);
        // Hint: header == null here
{code}


> Cassandra TWCS is not removing fully expired tables
> ---------------------------------------------------
>
>                 Key: CASSANDRA-13005
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-13005
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Compaction
>         Environment: Cassandra 3.0.9 / Casandra 3.9
> Java HotSpot(TM) 64-Bit Server VM version 25.112-b15 (Java version 1.8.0_112-b15)
> Linux 3.16
>            Reporter: Christian Esken
>         Attachments: sstablemetadata-empty-type-that-is-3GB.txt
>
>
> We have a table where all columns are stored with TTL of maximum 4 hours. Usually TWCS
compaction properly removes  expired data via tombstone compaction and also removes fully
expired tables. The number of SSTables is nearly constant since weeks. Good.
> The problem:  Suddenly TWCS does not remove old SSTables any longer. They are being recreated
frequently (judging form the file creation timestamp), but the number of tables is growing.
> - sstablemetadata shows strange data, as if the table is completely empty.
> - sstabledump throws an Exception when running it on such a SSTable



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message