Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 606E8200B8D for ; Fri, 23 Sep 2016 23:11:23 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 5EF5F160AD0; Fri, 23 Sep 2016 21:11:23 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 80648160ACF for ; Fri, 23 Sep 2016 23:11:22 +0200 (CEST) Received: (qmail 44227 invoked by uid 500); 23 Sep 2016 21:11:21 -0000 Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@cassandra.apache.org Delivered-To: mailing list commits@cassandra.apache.org Received: (qmail 44216 invoked by uid 99); 23 Sep 2016 21:11:21 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 23 Sep 2016 21:11:21 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id 7EE782C0B05 for ; Fri, 23 Sep 2016 21:11:21 +0000 (UTC) Date: Fri, 23 Sep 2016 21:11:21 +0000 (UTC) From: "Michael Shuler (JIRA)" To: commits@cassandra.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (CASSANDRA-12435) Cassandra timeout when using secondary index in cluster mode. MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Fri, 23 Sep 2016 21:11:23 -0000 [ https://issues.apache.org/jira/browse/CASSANDRA-12435?page=3Dcom.atl= assian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Shuler updated CASSANDRA-12435: --------------------------------------- Fix Version/s: (was: 3.5) 3.x > Cassandra timeout when using secondary index in cluster mode. > ------------------------------------------------------------- > > Key: CASSANDRA-12435 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1243= 5 > Project: Cassandra > Issue Type: Bug > Environment: C* 3.5 | java-driver|AWS g2.2xlarge|ubuntu 14.04 > Reporter: Shijian Liu > Fix For: 3.x > > > I have a 3-node cluster with a keyspace with RF=3D3, then I created a tab= le like below and inserted some data(about 10MB). > CREATE TABLE vehicle_fuzzy_plate ( > day int, > hour int, > repo int, > sensor_id int, > ts timestamp, > id timeuuid, > plate_text_pattern_70 text, > plate_text_pattern_10 text, > plate_text_pattern_11 text, > plate_text_pattern_12 text, > plate_text_pattern_13 text, > plate_text_pattern_14 text, > plate_text_pattern_15 text, > plate_text_pattern_16 text, > plate_text_pattern_20 text, > plate_text_pattern_21 text, > plate_text_pattern_22 text, > plate_text_pattern_23 text, > plate_text_pattern_24 text, > plate_text_pattern_25 text, > plate_text_pattern_30 text, > plate_text_pattern_31 text, > plate_text_pattern_32 text, > plate_text_pattern_33 text, > plate_text_pattern_34 text, > plate_text_pattern_40 text, > plate_text_pattern_41 text, > plate_text_pattern_42 text, > plate_text_pattern_43 text, > plate_text_pattern_50 text, > plate_text_pattern_51 text, > plate_text_pattern_52 text, > plate_text_pattern_60 text, > plate_text_pattern_61 text, > PRIMARY KEY (day, repo, sensor_id, ts, id) > ) WITH CLUSTERING ORDER BY (repo ASC, sensor_id ASC, ts ASC, id ASC); > CREATE INDEX plate_text_pattern_10_idx ON vehicle_fuzzy_plate (plate_text= _pattern_10); > CREATE INDEX plate_text_pattern_11_idx ON vehicle_fuzzy_plate (plate_text= _pattern_11); > CREATE INDEX plate_text_pattern_12_idx ON vehicle_fuzzy_plate (plate_text= _pattern_12); > CREATE INDEX plate_text_pattern_13_idx ON vehicle_fuzzy_plate (plate_text= _pattern_13); > CREATE INDEX plate_text_pattern_14_idx ON vehicle_fuzzy_plate (plate_text= _pattern_14); > CREATE INDEX plate_text_pattern_15_idx ON vehicle_fuzzy_plate (plate_text= _pattern_15); > CREATE INDEX plate_text_pattern_16_idx ON vehicle_fuzzy_plate (plate_text= _pattern_16); > CREATE INDEX plate_text_pattern_20_idx ON vehicle_fuzzy_plate (plate_text= _pattern_20); > CREATE INDEX plate_text_pattern_21_idx ON vehicle_fuzzy_plate (plate_text= _pattern_21); > CREATE INDEX plate_text_pattern_22_idx ON vehicle_fuzzy_plate (plate_text= _pattern_22); > CREATE INDEX plate_text_pattern_23_idx ON vehicle_fuzzy_plate (plate_text= _pattern_23); > CREATE INDEX plate_text_pattern_24_idx ON vehicle_fuzzy_plate (plate_text= _pattern_24); > CREATE INDEX plate_text_pattern_25_idx ON vehicle_fuzzy_plate (plate_text= _pattern_25); > CREATE INDEX plate_text_pattern_30_idx ON vehicle_fuzzy_plate (plate_text= _pattern_30); > CREATE INDEX plate_text_pattern_31_idx ON vehicle_fuzzy_plate (plate_text= _pattern_31); > CREATE INDEX plate_text_pattern_32_idx ON vehicle_fuzzy_plate (plate_text= _pattern_32); > CREATE INDEX plate_text_pattern_33_idx ON vehicle_fuzzy_plate (plate_text= _pattern_33); > CREATE INDEX plate_text_pattern_34_idx ON vehicle_fuzzy_plate (plate_text= _pattern_34); > CREATE INDEX plate_text_pattern_40_idx ON vehicle_fuzzy_plate (plate_text= _pattern_40); > CREATE INDEX plate_text_pattern_41_idx ON vehicle_fuzzy_plate (plate_text= _pattern_41); > CREATE INDEX plate_text_pattern_42_idx ON vehicle_fuzzy_plate (plate_text= _pattern_42); > CREATE INDEX plate_text_pattern_43_idx ON vehicle_fuzzy_plate (plate_text= _pattern_43); > CREATE INDEX plate_text_pattern_50_idx ON vehicle_fuzzy_plate (plate_text= _pattern_50); > CREATE INDEX plate_text_pattern_51_idx ON vehicle_fuzzy_plate (plate_text= _pattern_51); > CREATE INDEX plate_text_pattern_52_idx ON vehicle_fuzzy_plate (plate_text= _pattern_52); > CREATE INDEX plate_text_pattern_60_idx ON vehicle_fuzzy_plate (plate_text= _pattern_60); > CREATE INDEX plate_text_pattern_61_idx ON vehicle_fuzzy_plate (plate_text= _pattern_61); > CREATE INDEX plate_text_pattern_70_idx ON vehicle_fuzzy_plate (plate_text= _pattern_70); > After that, I send some queries through Java driver with Consistency=3DQu= orum one by one, for example, > SELECT id,sensor_id,ts FROM vehicle_fuzzy_plate WHERE day IN (16993,16994= ,16995,16996,16997,16998,16999,17000,17001,17002,17003,17004,17005,17006,17= 007,17008,17009,17010,17011,17012,17013,17014,17015,17016,17017,17018,17019= ,17020,17021,17022,17023) AND repo IN (234) AND sensor_id IN (123,16) AND t= s>=3D1468226847000 AND ts<=3D1470818847000 AND plate_text_pattern_70 IN ('7= 0=E8=B4=B5JBP378') ALLOW FILTERING; > The weird part is that some queries will response timeout (about 30%). I = checked the debug.log and found these exception, > DEBUG [SharedPool-Worker-1] 2016-08-11 14:09:00,885 ReadCallback.java:126= - Timed out; received 1 of 2 responses (including data) > ERROR [MessagingService-Incoming-/192.168.2.122] 2016-08-11 14:09:05,954 = CassandraDaemon.java:217 - Exception in thread Thread[MessagingService-Inco= ming-/192.168.2.122,5,main] > java.lang.ArrayIndexOutOfBoundsException: 74 > at org.apache.cassandra.db.filter.AbstractClusteringIndexFilter$F= ilterSerializer.deserialize(AbstractClusteringIndexFilter.java:74) ~[apache= -cassandra-3.5.jar:3.5] > at org.apache.cassandra.db.SinglePartitionReadCommand$Deserialize= r.deserialize(SinglePartitionReadCommand.java:1045) ~[apache-cassandra-3.5.= jar:3.5] > at org.apache.cassandra.db.ReadCommand$Serializer.deserialize(Rea= dCommand.java:667) ~[apache-cassandra-3.5.jar:3.5] > at org.apache.cassandra.db.ReadCommand$Serializer.deserialize(Rea= dCommand.java:595) ~[apache-cassandra-3.5.jar:3.5] > at org.apache.cassandra.net.MessageIn.read(MessageIn.java:114) ~[= apache-cassandra-3.5.jar:3.5] > at org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(= IncomingTcpConnection.java:190) ~[apache-cassandra-3.5.jar:3.5] > at org.apache.cassandra.net.IncomingTcpConnection.receiveMessages= (IncomingTcpConnection.java:178) ~[apache-cassandra-3.5.jar:3.5] > at org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcp= Connection.java:92) ~[apache-cassandra-3.5.jar:3.5] > I try to clean table and send the query, but timeout comes out occasional= ly too. > However, queries will work fine if I don't use the secondary index, just = like this, > SELECT id,sensor_id,ts FROM vehicle_fuzzy_plate WHERE day IN (16993,16994= ,16995,16996,16997,16998,16999,17000,17001,17002,17003,17004,17005,17006,17= 007,17008,17009,17010,17011,17012,17013,17014,17015,17016,17017,17018,17019= ,17020,17021,17022,17023) AND repo IN (234) AND sensor_id IN (123,16) AND t= s>=3D1468226847000 AND ts<=3D1470818847000; > Everything works fine on a single node with RF=3D1 with more than 10GB da= ta. -- This message was sent by Atlassian JIRA (v6.3.4#6332)