Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id E5B45200BCF for ; Mon, 5 Dec 2016 16:22:00 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id E44B8160AF9; Mon, 5 Dec 2016 15:22:00 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 11F33160B21 for ; Mon, 5 Dec 2016 16:21:59 +0100 (CET) Received: (qmail 55338 invoked by uid 500); 5 Dec 2016 15:21:59 -0000 Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@cassandra.apache.org Delivered-To: mailing list commits@cassandra.apache.org Received: (qmail 55310 invoked by uid 99); 5 Dec 2016 15:21:58 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 05 Dec 2016 15:21:58 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id B58002C1F5A for ; Mon, 5 Dec 2016 15:21:58 +0000 (UTC) Date: Mon, 5 Dec 2016 15:21:58 +0000 (UTC) From: "Brice Dutheil (JIRA)" To: commits@cassandra.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (CASSANDRA-12620) Resurrected empty rows on update to 3.x MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Mon, 05 Dec 2016 15:22:01 -0000 [ https://issues.apache.org/jira/browse/CASSANDRA-12620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15722517#comment-15722517 ] Brice Dutheil commented on CASSANDRA-12620: ------------------------------------------- Hi, We just had the same problem, upgrading from DSE 4.8.6 to DSE 5.0.4. The problem has been identified on 3 different environments. Dead / tombstoned rows with an expired TTL show up after the upgrade. The thing is only the primary key is non null, all other fields are null. We only insert non null values using TTL that have various TTL from 60s to 1day. {code} select * from ttl_entries where key='d7c10084-724c-4117-9927-b927a972203b'; key | field1 | field2 | field3 | field4 | field5 | field6 | field7 | field8 --------------------------------------+----------+-----------+-----------+----------+----------+--------------+--------+----------- d7c10084-724c-4117-9927-b927a972203b | null | null | null | null | null | null | null | null {code} Here's an {{sstabledump}} of one of the impacted row. Note that the {{liveness_info}} is missing TTL informations: {code:title=sstabledump row d7c10084-724c-4117-9927-b927a972203b} { "partition" : { "key" : [ "d7c10084-724c-4117-9927-b927a972203b" ], "position" : 6557684 }, "rows" : [ { "type" : "row", "position" : 6557734, "liveness_info" : { "tstamp" : "2016-11-25T02:24:06.237Z" }, "cells" : [ { "name" : "field1", "deletion_info" : { "local_delete_time" : "2016-11-25T02:24:06Z" } }, { "name" : "field2", "deletion_info" : { "local_delete_time" : "2016-11-25T02:24:06Z" } }, { "name" : "field3", "deletion_info" : { "local_delete_time" : "2016-11-25T02:24:06Z" } }, { "name" : "field4", "deletion_info" : { "local_delete_time" : "2016-11-25T02:24:06Z" } }, { "name" : "field5", "deletion_info" : { "local_delete_time" : "2016-11-25T02:24:06Z" } }, { "name" : "field6", "deletion_info" : { "local_delete_time" : "2016-11-25T02:24:06Z" } }, { "name" : "field8", "deletion_info" : { "local_delete_time" : "2016-11-25T02:24:06Z" } }, { "name" : "field7", "deletion_info" : { "marked_deleted" : "2016-11-25T02:24:06.236999Z", "local_delete_time" : "2016-11-25T02:24:06Z" } }, { "name" : "field7", "path" : [ "VALUE_01" ], "deletion_info" : { "local_delete_time" : "2016-11-25T02:24:06Z" } }, { "name" : "field7", "path" : [ "VALUE_02" ], "deletion_info" : { "local_delete_time" : "2016-11-25T02:24:06Z" } }, { "name" : "field7", "path" : [ "VALUE_03" ], "deletion_info" : { "local_delete_time" : "2016-11-25T02:24:06Z" } }, { "name" : "field7", "path" : [ "VALUE_04" ], "deletion_info" : { "local_delete_time" : "2016-11-25T02:24:06Z" } }, { "name" : "field7", "path" : [ "VALUE_05" ], "deletion_info" : { "local_delete_time" : "2016-11-25T02:24:06Z" } }, { "name" : "field7", "path" : [ "VALUE_06" ], "deletion_info" : { "local_delete_time" : "2016-11-25T02:24:06Z" } }, { "name" : "field7", "path" : [ "VALUE_07" ], "deletion_info" : { "local_delete_time" : "2016-11-25T02:24:06Z" } }, { "name" : "field7", "path" : [ "VALUE_08" ], "deletion_info" : { "local_delete_time" : "2016-11-25T02:24:06Z" } }, { "name" : "field7", "path" : [ "VALUE_09" ], "deletion_info" : { "local_delete_time" : "2016-11-25T02:24:06Z" } }, { "name" : "field7", "path" : [ "VALUE_10" ], "deletion_info" : { "local_delete_time" : "2016-11-25T02:24:06Z" } }, { "name" : "field7", "path" : [ "VALUE_11" ], "deletion_info" : { "local_delete_time" : "2016-11-25T02:24:06Z" } }, { "name" : "field7", "path" : [ "VALUE_12" ], "deletion_info" : { "local_delete_time" : "2016-11-25T02:24:06Z" } }, { "name" : "field7", "path" : [ "VALUE_13" ], "deletion_info" : { "local_delete_time" : "2016-11-25T02:24:06Z" } }, { "name" : "field7", "path" : [ "VALUE_14" ], "deletion_info" : { "local_delete_time" : "2016-11-25T02:24:06Z" } }, { "name" : "field7", "path" : [ "VALUE_15" ], "deletion_info" : { "local_delete_time" : "2016-11-25T02:24:06Z" } }, { "name" : "field7", "path" : [ "VALUE_16" ], "deletion_info" : { "local_delete_time" : "2016-11-25T02:24:06Z" } }, { "name" : "field7", "path" : [ "VALUE_17" ], "deletion_info" : { "local_delete_time" : "2016-11-25T02:24:06Z" } }, { "name" : "field7", "path" : [ "VALUE_18" ], "deletion_info" : { "local_delete_time" : "2016-11-25T02:24:06Z" } }, { "name" : "field7", "path" : [ "VALUE_19" ], "deletion_info" : { "local_delete_time" : "2016-11-25T02:24:06Z" } }, { "name" : "field7", "path" : [ "VALUE_20" ], "deletion_info" : { "local_delete_time" : "2016-11-25T02:24:06Z" } }, { "name" : "field7", "path" : [ "VALUE_21" ], "deletion_info" : { "local_delete_time" : "2016-11-25T02:24:06Z" } }, { "name" : "field7", "path" : [ "VALUE_22" ], "deletion_info" : { "local_delete_time" : "2016-11-25T02:24:06Z" } }, { "name" : "field7", "path" : [ "VALUE_23" ], "deletion_info" : { "local_delete_time" : "2016-11-25T02:24:06Z" } }, { "name" : "field7", "path" : [ "VALUE_24" ], "deletion_info" : { "local_delete_time" : "2016-11-25T02:24:06Z" } }, { "name" : "field7", "path" : [ "VALUE_25" ], "deletion_info" : { "local_delete_time" : "2016-11-25T02:24:06Z" } } ] } ] }, {code} Data inserted after the upgrade with TTL have correct TTL metadata: {code} "liveness_info" : { "tstamp" : "2016-11-29T09:43:24.953Z", "ttl" : 3600, "expires_at" : "2016-11-29T10:43:24Z", "expired" : true } {code} Here's the describe table. {code:title=table description after the upgrade} CREATE TABLE ttl_entries ( key text PRIMARY KEY, field1 text, field2 int, field3 text, field4 bigint, field5 int, field6 text, field7 set, field8 text ) WITH bloom_filter_fp_chance = 0.01 AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'} AND compaction = {'class': 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 'max_threshold': '32', 'min_threshold': '4'} AND compression = {'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.SnappyCompressor'} AND crc_check_chance = 1.0 AND dclocal_read_repair_chance = 0.0 AND default_time_to_live = 0 AND gc_grace_seconds = 864000 AND max_index_interval = 2048 AND memtable_flush_period_in_ms = 0 AND min_index_interval = 128 AND read_repair_chance = 0.1 AND speculative_retry = '99PERCENTILE'; {code} Our upgrade procedure : For each node before upgrading * nodetool repair -pr Then on each node * nodetool drain * stop the node * change files config * link to new binaries * start the nodes * nodetool upgradesstables PS : I think the title should be edited to indicates that it's rows with _expired TTL_ that shows up. > Resurrected empty rows on update to 3.x > --------------------------------------- > > Key: CASSANDRA-12620 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12620 > Project: Cassandra > Issue Type: Bug > Reporter: Collin Sauve > Assignee: Benjamin Lerer > > We had the below table on C* 2.x (dse 4.8.4, we assume was 2.1.15.1423 according to documentation), and were entering TTLs at write-time using the DataStax C# Driver (using the POCO mapper). > Upon upgrade to 3.0.8.1293 (DSE 5.0.2), we are seeing a lot of rows that: > * should have been TTL'd > * have no non-primary-key column data > {code} > CREATE TABLE applicationservices.aggregate_bucket_event_v3 ( > bucket_type int, > bucket_id text, > date timestamp, > aggregate_id text, > event_type int, > event_id text, > entities list>>, > identity_sid text, > PRIMARY KEY ((bucket_type, bucket_id), date, aggregate_id, event_type, event_id) > ) WITH CLUSTERING ORDER BY (date DESC, aggregate_id ASC, event_type ASC, event_id ASC) > AND bloom_filter_fp_chance = 0.1 > AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'} > AND comment = '' > AND compaction = {'class': 'org.apache.cassandra.db.compaction.LeveledCompactionStrategy'} > AND compression = {'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'} > AND crc_check_chance = 1.0 > AND dclocal_read_repair_chance = 0.1 > AND default_time_to_live = 0 > AND gc_grace_seconds = 864000 > AND max_index_interval = 2048 > AND memtable_flush_period_in_ms = 0 > AND min_index_interval = 128 > AND read_repair_chance = 0.0 > AND speculative_retry = '99PERCENTILE'; > {code} > {code} > { > "partition" : { > "key" : [ "0", "26492" ], > "position" : 54397932 > }, > "rows" : [ > { > "type" : "row", > "position" : 54397961, > "clustering" : [ "2016-09-07 23:33Z", "3651664", "0", "773665449947099136" ], > "liveness_info" : { "tstamp" : "2016-09-07T23:34:09.758Z", "ttl" : 172741, "expires_at" : "2016-09-09T23:33:10Z", "expired" : false }, > "cells" : [ > { "name" : "identity_sid", "value" : "p_tw_zahidana" }, > { "name" : "entities", "deletion_info" : { "marked_deleted" : "2016-09-07T23:34:09.757999Z", "local_delete_time" : "2016-09-07T23:34:09Z" } }, > { "name" : "entities", "path" : [ "936e17e1-7553-11e6-9b92-29a33b5827c3" ], "value" : "0:https\\://www.youtube.com/watch?v=pwAJAssv6As" }, > { "name" : "entities", "path" : [ "936e17e2-7553-11e6-9b92-29a33b5827c3" ], "value" : "2:youtube" } > ] > }, > { > "type" : "row", > }, > { > "type" : "row", > "position" : 54397177, > "clustering" : [ "2016-08-17 10:00Z", "6387376", "0", "765850666296225792" ], > "liveness_info" : { "tstamp" : "2016-08-17T11:26:15.917001Z" }, > "cells" : [ ] > }, > { > "type" : "row", > "position" : 54397227, > "clustering" : [ "2016-08-17 07:00Z", "6387376", "0", "765805367347601409" ], > "liveness_info" : { "tstamp" : "2016-08-17T08:11:17.587Z" }, > "cells" : [ ] > }, > { > "type" : "row", > "position" : 54397276, > "clustering" : [ "2016-08-17 04:00Z", "6387376", "0", "765760069858365441" ], > "liveness_info" : { "tstamp" : "2016-08-17T05:58:11.228Z" }, > "cells" : [ ] > }, > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)