Return-Path: Delivered-To: apmail-incubator-cassandra-commits-archive@minotaur.apache.org Received: (qmail 65186 invoked from network); 20 Jan 2010 23:27:15 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 20 Jan 2010 23:27:15 -0000 Received: (qmail 49005 invoked by uid 500); 20 Jan 2010 23:27:15 -0000 Delivered-To: apmail-incubator-cassandra-commits-archive@incubator.apache.org Received: (qmail 48987 invoked by uid 500); 20 Jan 2010 23:27:15 -0000 Mailing-List: contact cassandra-commits-help@incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: cassandra-dev@incubator.apache.org Delivered-To: mailing list cassandra-commits@incubator.apache.org Received: (qmail 48977 invoked by uid 99); 20 Jan 2010 23:27:15 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 20 Jan 2010 23:27:15 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [140.211.11.140] (HELO brutus.apache.org) (140.211.11.140) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 20 Jan 2010 23:27:14 +0000 Received: from brutus.apache.org (localhost [127.0.0.1]) by brutus.apache.org (Postfix) with ESMTP id 85F63234C1E9 for ; Wed, 20 Jan 2010 15:26:54 -0800 (PST) Message-ID: <169746288.381681264030014547.JavaMail.jira@brutus.apache.org> Date: Wed, 20 Jan 2010 23:26:54 +0000 (UTC) From: "Jonathan Ellis (JIRA)" To: cassandra-commits@incubator.apache.org Subject: [jira] Updated: (CASSANDRA-724) Insert/Get Contention In-Reply-To: <597471691.364581263976078116.JavaMail.jira@brutus.apache.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/CASSANDRA-724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-724: ------------------------------------- Attachment: debug.patch patch to add debug timing info if you want to investigate further. there does seem to be occasional latency spikes inside ColumnFamilyStore.apply that I do not yet understand. when cpus are busy w/ compaction latency increases. no real surprise there. thrift sometimes adds 10s of ms of latency according to the differences b/t what my python client sees and what CassandraServer sees. the java side of thrift does call setTcpNoDelay(true), but the python side does not -- the equivalent would be, setsockopt(SOL_TCP, TCP_NODELAY, 1). that is probably the culprit. > Insert/Get Contention > --------------------- > > Key: CASSANDRA-724 > URL: https://issues.apache.org/jira/browse/CASSANDRA-724 > Project: Cassandra > Issue Type: Bug > Components: Core > Reporter: Chris Goffinet > Assignee: Jonathan Ellis > Fix For: 0.6 > > Attachments: 724.patch, debug.patch, test_case.py > > > We tried out the socket io patch in CASSANDRA-705, tested the latest JVM of b18 for 1.6. Still seeing very strange insert times. We see this with get_slices as well but it's easy to reproduce with batch_insert. I wonder if its related to Memtable contention, it's pretty easy to see the slow times when you restart the test script attached. We are running this on a 7 node cluster, <1% cpu. Consistency Level of 1. > Results > --------------------- > Slow insert test.10882 0.203548192978 > Slow insert test.18005 0.203876972198 > Slow insert test.21154 0.204496860504 > Slow insert test.22054 0.0444049835205 > Slow insert test.26445 0.201545000076 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.