Return-Path: X-Original-To: apmail-cassandra-commits-archive@www.apache.org Delivered-To: apmail-cassandra-commits-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 50E57116EB for ; Wed, 16 Jul 2014 23:23:06 +0000 (UTC) Received: (qmail 48785 invoked by uid 500); 16 Jul 2014 23:23:05 -0000 Delivered-To: apmail-cassandra-commits-archive@cassandra.apache.org Received: (qmail 48702 invoked by uid 500); 16 Jul 2014 23:23:05 -0000 Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@cassandra.apache.org Delivered-To: mailing list commits@cassandra.apache.org Received: (qmail 48432 invoked by uid 99); 16 Jul 2014 23:23:05 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 16 Jul 2014 23:23:05 +0000 Date: Wed, 16 Jul 2014 23:23:05 +0000 (UTC) From: "Vladimir Avram (JIRA)" To: commits@cassandra.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Created] (CASSANDRA-7560) 'nodetool repair -pr' leads to indefinitely hanging AntiEntropySession MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 Vladimir Avram created CASSANDRA-7560: ----------------------------------------- Summary: 'nodetool repair -pr' leads to indefinitely hanging AntiEntropySession Key: CASSANDRA-7560 URL: https://issues.apache.org/jira/browse/CASSANDRA-7560 Project: Cassandra Issue Type: Bug Components: Core Reporter: Vladimir Avram Running {{nodetool repair -pr}} will sometimes hang on one of the resulting AntiEntropySessions. The system logs will show the repair command starting {panel} INFO [Thread-3079] 2014-07-15 02:22:56,514 StorageService.java (line 2569) Starting repair command #1, repairing 256 ranges for keyspace x {panel} You can then see a few AntiEntropySessions completing with: {panel} INFO [AntiEntropySessions:2] 2014-07-15 02:28:12,766 RepairSession.java (line 282) [repair #eefb3c30-0bc6-11e4-83f7-a378978d0c49] session completed successfully {panel} Finally we reach an AntiEntropySession at some point that hangs just before requesting the merkle trees for the next column family in line for repair. So we first see the previous CF being finished and the whole repair sessions hangs here with no visible progress or errors on this or any of the related nodes. {panel} INFO [AntiEntropyStage:1] 2014-07-15 02:38:20,325 RepairSession.java (line 221) [repair #8f85c1b0-0bc8-11e4-83f7-a378978d0c49] previous_cf is fully synced {panel} -- This message was sent by Atlassian JIRA (v6.2#6252)