Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 32726D78E for ; Mon, 17 Sep 2012 09:08:01 +0000 (UTC) Received: (qmail 10217 invoked by uid 500); 17 Sep 2012 09:07:58 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 10178 invoked by uid 500); 17 Sep 2012 09:07:58 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 10162 invoked by uid 99); 17 Sep 2012 09:07:58 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 17 Sep 2012 09:07:58 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=FSL_RCVD_USER,HTML_MESSAGE,NORMAL_HTTP_TO_IP,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of software.research.work@gmail.com designates 209.85.223.172 as permitted sender) Received: from [209.85.223.172] (HELO mail-ie0-f172.google.com) (209.85.223.172) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 17 Sep 2012 09:07:53 +0000 Received: by iec9 with SMTP id 9so1028536iec.31 for ; Mon, 17 Sep 2012 02:07:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; bh=b03NEjHyhMiGr/8gIas0G649f+KNYyOzNe/H78i34OU=; b=zp0r5zssQC8fXTKulqY1jVQ4kneHuoyR2csZ8odQ8a0eQguKxEMpO/mTsW010fXK71 n4ChgvsUX2lHf1HunVwEBAEg3/pkwPVtFVmtjv9xqfV3NQFdasUxt4wbMfXK1ZhgINrt +j2n0xriYhSYBZcx6rEW6J/Ew23YNkDsggYgIAIIy5w5uV9N3Y0zt2XBDYKR2s57rnMI HdgeAWyQeeONgfZ55ihHCUkqn1d2tEkbe2R4yxCSwqB7+04h772uFJDC584c6cB52EAK AgeNqwHz8xJ3XF7dYlhqx5ZnYrZG2TOy7oOqIp0iwNlQBewd8XDOEDU0nTbXtwmxlKmo j4Bg== Received: by 10.50.195.161 with SMTP id if1mr5941089igc.66.1347872852532; Mon, 17 Sep 2012 02:07:32 -0700 (PDT) MIME-Version: 1.0 Received: by 10.231.24.209 with HTTP; Mon, 17 Sep 2012 02:06:52 -0700 (PDT) In-Reply-To: References: From: B R Date: Mon, 17 Sep 2012 14:36:52 +0530 Message-ID: Subject: Re: Repair: Issue in netstats To: user@cassandra.apache.org Content-Type: multipart/alternative; boundary=14dae9340e1d93ddd204c9e21add X-Virus-Checked: Checked by ClamAV on apache.org --14dae9340e1d93ddd204c9e21add Content-Type: text/plain; charset=ISO-8859-1 Sorry for the delay; been out of the loop. Could this problem be due to running repair on a node upgraded to 1.0.11 but the other node in the cluster is still at 0.8.x ? On Fri, Sep 7, 2012 at 9:11 PM, Sylvain Lebresne wrote: > That obviously shouldn't happen and I don't remember any open ticket > related to that. You might want to open a ticket on jira > (https://issues.apache.org/jira/browse/CASSANDRA). > > -- > Sylvain > > On Fri, Sep 7, 2012 at 10:50 AM, B R > wrote: > > We have upgraded a 0.8 cluster to 1.0.11. After upgrading the first node > and > > running upgradesstables, we have run a routine repair operation, This > > operation has been running for a long time and does not seem to be > > progressing. > > > > Running netstats has shown unexpected values for percentages as shown > below. > > Any clue as to what could be be issue ? > > > > bin/nodetool -h 172.16.0.34 netstats > > Mode: NORMAL > > Streaming to: /172.16.0.29 > > /data/cassandra/data/Keyspace1/Standard1-hd-16609-Data.db sections=116 > > progress=19946657796608/334406146 - 5964800% > > /data/cassandra/data/Keyspace1/Standard1-hd-16618-Data.db sections=116 > > progress=0/179880575 - 0% > > /data/cassandra/data/Keyspace1/Standard1-hd-16620-Data.db sections=12 > > progress=0/1448134 - 0% > > /data/cassandra/data/Keyspace1/Standard1-hd-16616-Data.db sections=116 > > progress=0/350403675 - 0% > > /data/cassandra/data/Keyspace1/Standard1-hd-16602-Data.db sections=89 > > progress=0/27569594 - 0% > > /data/cassandra/data/Keyspace1/Standard1-hd-16615-Data.db sections=1 > > progress=0/95043 - 0% > > /data/cassandra/data/Keyspace1/Standard1-hd-16617-Data.db sections=1 > > progress=0/232800 - 0% > > /data/cassandra/data/Keyspace1/Standard1-hd-16612-Data.db sections=1 > > progress=0/82705 - 0% > > /data/cassandra/data/Keyspace1/Standard1-hd-16603-Data.db sections=116 > > progress=0/724836994 - 0% > > /data/cassandra/data/Keyspace1/Standard1-hd-16607-Data.db sections=116 > > progress=0/401797714 - 0% > > /data/cassandra/data/Keyspace1/Standard1-hd-16608-Data.db sections=2 > > progress=0/301297 - 0% > > /data/cassandra/data/Keyspace1/Standard1-hd-16619-Data.db sections=3 > > progress=0/829914 - 0% > > /data/cassandra/data/Keyspace1/Standard1-hd-16604-Data.db sections=2 > > progress=0/288460 - 0% > > /data/cassandra/data/Keyspace1/Standard1-hd-16610-Data.db sections=13 > > progress=0/1954639 - 0% > > /data/cassandra/data/Keyspace1/Standard1-hd-16606-Data.db sections=8 > > progress=0/1187649 - 0% > > /data/cassandra/data/Keyspace1/Standard1-hd-16613-Data.db sections=1 > > progress=0/141714 - 0% > > /data/cassandra/data/Keyspace1/Standard1-hd-16614-Data.db sections=116 > > progress=0/390168999 - 0% > > /data/cassandra/data/Keyspace1/Standard1-hd-16609-Data.db sections=111 > > progress=13620592201686/303748754 - 4484163% > > /data/cassandra/data/Keyspace1/Standard1-hd-16618-Data.db sections=110 > > progress=0/162808076 - 0% > > /data/cassandra/data/Keyspace1/Standard1-hd-16620-Data.db sections=10 > > progress=0/1922996 - 0% > > /data/cassandra/data/Keyspace1/Standard1-hd-16616-Data.db sections=111 > > progress=0/350744309 - 0% > > /data/cassandra/data/Keyspace1/Standard1-hd-16602-Data.db sections=87 > > progress=0/24364920 - 0% > > /data/cassandra/data/Keyspace1/Standard1-hd-16615-Data.db sections=2 > > progress=0/228764 - 0% > > /data/cassandra/data/Keyspace1/Standard1-hd-16603-Data.db sections=111 > > progress=0/720722886 - 0% > > /data/cassandra/data/Keyspace1/Standard1-hd-16607-Data.db sections=111 > > progress=0/364643588 - 0% > > /data/cassandra/data/Keyspace1/Standard1-hd-16608-Data.db sections=4 > > progress=0/963207 - 0% > > /data/cassandra/data/Keyspace1/Standard1-hd-16619-Data.db sections=2 > > progress=0/360024 - 0% > > /data/cassandra/data/Keyspace1/Standard1-hd-16604-Data.db sections=1 > > progress=0/72842 - 0% > > /data/cassandra/data/Keyspace1/Standard1-hd-16610-Data.db sections=11 > > progress=0/1381176 - 0% > > /data/cassandra/data/Keyspace1/Standard1-hd-16606-Data.db sections=13 > > progress=0/3266736 - 0% > > /data/cassandra/data/Keyspace1/Standard1-hd-16613-Data.db sections=2 > > progress=0/639705 - 0% > > /data/cassandra/data/Keyspace1/Standard1-hd-16614-Data.db sections=111 > > progress=0/358443928 - 0% > > Nothing streaming from /172.16.0.29 > > Pool Name Active Pending Completed > > Commands n/a 0 19 > > Responses n/a 0 2050444 > > > > Regards. > --14dae9340e1d93ddd204c9e21add Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Sorry for the delay; been out of the loop.

Could this pr= oblem be due to running repair on a node upgraded to 1.0.11 but the other n= ode in the cluster is still at 0.8.x ?

On Fri, Sep 7, 2012 at 9:11 PM, Sylvain Lebresne <sylvain@datastax.com<= /a>> wrote:
That obviously shouldn't happen and I don't remember any open ticke= t
related to that. You might want to open a ticket on jira
(https://issues.apache.org/jira/browse/CASSANDRA).

--
Sylvain

On Fri, Sep 7, 2012 at 10:50 AM, B R <software.research.work@gmail.com> wrote:
> We have upgraded a 0.8 cluster to 1.0.11. After upgrading the first no= de and
> running upgradesstables, we have run a routine repair operation, This<= br> > operation has been running for a long time and does not seem to be
> progressing.
>
> Running netstats has shown unexpected values for percentages as shown = below.
> Any clue as to what could be be issue ?
>
> bin/nodetool -h 172.16.0.34 netstats
> Mode: NORMAL
> Streaming to: /172.16= .0.29
> =A0 =A0/data/cassandra/data/Keyspace1/Standard1-hd-16609-Data.db secti= ons=3D116
> progress=3D19946657796608/334406146 - 5964800%
> =A0 =A0/data/cassandra/data/Keyspace1/Standard1-hd-16618-Data.db secti= ons=3D116
> progress=3D0/179880575 - 0%
> =A0 =A0/data/cassandra/data/Keyspace1/Standard1-hd-16620-Data.db secti= ons=3D12
> progress=3D0/1448134 - 0%
> =A0 =A0/data/cassandra/data/Keyspace1/Standard1-hd-16616-Data.db secti= ons=3D116
> progress=3D0/350403675 - 0%
> =A0 =A0/data/cassandra/data/Keyspace1/Standard1-hd-16602-Data.db secti= ons=3D89
> progress=3D0/27569594 - 0%
> =A0 =A0/data/cassandra/data/Keyspace1/Standard1-hd-16615-Data.db secti= ons=3D1
> progress=3D0/95043 - 0%
> =A0 =A0/data/cassandra/data/Keyspace1/Standard1-hd-16617-Data.db secti= ons=3D1
> progress=3D0/232800 - 0%
> =A0 =A0/data/cassandra/data/Keyspace1/Standard1-hd-16612-Data.db secti= ons=3D1
> progress=3D0/82705 - 0%
> =A0 =A0/data/cassandra/data/Keyspace1/Standard1-hd-16603-Data.db secti= ons=3D116
> progress=3D0/724836994 - 0%
> =A0 =A0/data/cassandra/data/Keyspace1/Standard1-hd-16607-Data.db secti= ons=3D116
> progress=3D0/401797714 - 0%
> =A0 =A0/data/cassandra/data/Keyspace1/Standard1-hd-16608-Data.db secti= ons=3D2
> progress=3D0/301297 - 0%
> =A0 =A0/data/cassandra/data/Keyspace1/Standard1-hd-16619-Data.db secti= ons=3D3
> progress=3D0/829914 - 0%
> =A0 =A0/data/cassandra/data/Keyspace1/Standard1-hd-16604-Data.db secti= ons=3D2
> progress=3D0/288460 - 0%
> =A0 =A0/data/cassandra/data/Keyspace1/Standard1-hd-16610-Data.db secti= ons=3D13
> progress=3D0/1954639 - 0%
> =A0 =A0/data/cassandra/data/Keyspace1/Standard1-hd-16606-Data.db secti= ons=3D8
> progress=3D0/1187649 - 0%
> =A0 =A0/data/cassandra/data/Keyspace1/Standard1-hd-16613-Data.db secti= ons=3D1
> progress=3D0/141714 - 0%
> =A0 =A0/data/cassandra/data/Keyspace1/Standard1-hd-16614-Data.db secti= ons=3D116
> progress=3D0/390168999 - 0%
> =A0 =A0/data/cassandra/data/Keyspace1/Standard1-hd-16609-Data.db secti= ons=3D111
> progress=3D13620592201686/303748754 - 4484163%
> =A0 =A0/data/cassandra/data/Keyspace1/Standard1-hd-16618-Data.db secti= ons=3D110
> progress=3D0/162808076 - 0%
> =A0 =A0/data/cassandra/data/Keyspace1/Standard1-hd-16620-Data.db secti= ons=3D10
> progress=3D0/1922996 - 0%
> =A0 =A0/data/cassandra/data/Keyspace1/Standard1-hd-16616-Data.db secti= ons=3D111
> progress=3D0/350744309 - 0%
> =A0 =A0/data/cassandra/data/Keyspace1/Standard1-hd-16602-Data.db secti= ons=3D87
> progress=3D0/24364920 - 0%
> =A0 =A0/data/cassandra/data/Keyspace1/Standard1-hd-16615-Data.db secti= ons=3D2
> progress=3D0/228764 - 0%
> =A0 =A0/data/cassandra/data/Keyspace1/Standard1-hd-16603-Data.db secti= ons=3D111
> progress=3D0/720722886 - 0%
> =A0 =A0/data/cassandra/data/Keyspace1/Standard1-hd-16607-Data.db secti= ons=3D111
> progress=3D0/364643588 - 0%
> =A0 =A0/data/cassandra/data/Keyspace1/Standard1-hd-16608-Data.db secti= ons=3D4
> progress=3D0/963207 - 0%
> =A0 =A0/data/cassandra/data/Keyspace1/Standard1-hd-16619-Data.db secti= ons=3D2
> progress=3D0/360024 - 0%
> =A0 =A0/data/cassandra/data/Keyspace1/Standard1-hd-16604-Data.db secti= ons=3D1
> progress=3D0/72842 - 0%
> =A0 =A0/data/cassandra/data/Keyspace1/Standard1-hd-16610-Data.db secti= ons=3D11
> progress=3D0/1381176 - 0%
> =A0 =A0/data/cassandra/data/Keyspace1/Standard1-hd-16606-Data.db secti= ons=3D13
> progress=3D0/3266736 - 0%
> =A0 =A0/data/cassandra/data/Keyspace1/Standard1-hd-16613-Data.db secti= ons=3D2
> progress=3D0/639705 - 0%
> =A0 =A0/data/cassandra/data/Keyspace1/Standard1-hd-16614-Data.db secti= ons=3D111
> progress=3D0/358443928 - 0%
> =A0Nothing streaming from /172.16.0.29
> Pool Name =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0Active =A0 Pending = =A0 =A0 =A0Completed
> Commands =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0n/a =A0 =A0 = =A0 =A0 0 =A0 =A0 =A0 =A0 =A0 =A0 19
> Responses =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 n/a =A0 =A0 =A0 = =A0 0 =A0 =A0 =A0 =A02050444
>
> Regards.

--14dae9340e1d93ddd204c9e21add--