Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 40AE53476 for ; Sun, 8 May 2011 21:38:23 +0000 (UTC) Received: (qmail 5667 invoked by uid 500); 8 May 2011 21:38:21 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 5639 invoked by uid 500); 8 May 2011 21:38:21 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 5631 invoked by uid 99); 8 May 2011 21:38:21 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 08 May 2011 21:38:21 +0000 X-ASF-Spam-Status: No, hits=-0.7 required=5.0 tests=FREEMAIL_FROM,RCVD_IN_DNSWL_LOW,RFC_ABUSE_POST,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of etamme@gmail.com designates 209.85.161.44 as permitted sender) Received: from [209.85.161.44] (HELO mail-fx0-f44.google.com) (209.85.161.44) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 08 May 2011 21:38:13 +0000 Received: by fxm15 with SMTP id 15so3831188fxm.31 for ; Sun, 08 May 2011 14:37:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:date:message-id:subject:from:to :content-type; bh=EMa7UsRYDz57PnxpRewlVW6XtmZpsU5JyCkwoUiyFUc=; b=QGJM034aajFtEsplTuFuNKs/TcCEPBV6E2GgmA5hxUvLkw1DCQRanAvHHdWxQV6Du0 yW9Lu/u+OYIE7KJFuGTjmFx9KMgUB0okwhLzjdqrUZwWyckhmoH5vfC+lQo6/K/FzT9p AIYhB8v7qZnMSoeki0wU36DEUdHnD3BvRKiO0= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=NASEVyr4ugbMJiytIV7MdQawBvAbOuItR7tf6Ts2Wb0Cg3+2beoo4OxdadDTWLzL7E exHYToWvPQ/QlAxKbpiXSsNdRbYRMfVgf9a/flyTFCQD5ibD1E4TaRWsd2DazFVTxEQq iSkq2+nPJUfetZ8rUVYKkrMMOinjnJLb5PAF4= MIME-Version: 1.0 Received: by 10.223.106.78 with SMTP id w14mr227403fao.35.1304890673564; Sun, 08 May 2011 14:37:53 -0700 (PDT) Received: by 10.223.98.196 with HTTP; Sun, 8 May 2011 14:37:53 -0700 (PDT) Date: Sun, 8 May 2011 17:37:53 -0400 Message-ID: Subject: RequestResponseStage Assertion Error From: Eric tamme To: user@cassandra.apache.org Content-Type: text/plain; charset=ISO-8859-1 X-Virus-Checked: Checked by ClamAV on apache.org I have a 4 node ring that was setup with tokens a,b,c,d using NTS and 2 nodes in each of 2 datacenters with a replication of DC1:1, DC2:1. I was getting uneven replica placement so I did a drop keyspace, followed by a nodetool move to DC1 having tokens (a,b) and DC2 having tokens (a+1,b+1) , then I removed the old data directory and recreated the keyspace. This has resolved my uneven replication, but now on one of my nodes I consistently get these errors. ERROR [RequestResponseStage:1] 2011-05-08 20:52:28,824 DebuggableThreadPoolExecutor.java (line 103) Error in ThreadPoolExecutor java.lang.AssertionError at org.apache.cassandra.service.ReadCallback.response(ReadCallback.java:127) at org.apache.cassandra.net.ResponseVerbHandler.doVerb(ResponseVerbHandler.java:49) at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:72) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:636) I dont know if i ... shutdown the node in the middle of one of the earlier operations or what. It seems to insert data fine, and my distribution is very even. What is this error, what is causing it, and how do i fix it? Thanks -Eric