Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: pass (nike.apache.org: domain of gulrich@netflix.com designates
 69.53.237.162 as permitted sender)
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws;
  s=s1024;d=netflix.com;
  h=from:to:subject:date:message-id:references:in-reply-to:content-type
    :mime-version;
  b=UyLNraSK4VIv03m1IuZ3NTLO+4hMMwxTqLPTX2mh3Sf2aOk9/5snNGFSqaEA/nr5FPH1wGTt
    17xtG0mD2iS9zRJr/C7tn9VEIs7sFOfnf4l58QIzyaM4HEjHcCN8wjhjRMPnPfUTNsga6c5U
    zZy8v25DGic3qpuM3v8s/74TBHQ=
From: Gregg Ulrich <gulrich@netflix.com>
To: "<user@cassandra.apache.org>" <user@cassandra.apache.org>
Subject: Re: nodetool repair hanging
Thread-Topic: nodetool repair hanging
Thread-Index: AQHNImUMUIkW4SnS7EGvLaNd2HNQ/ZasJq+A
Date: Wed, 25 Apr 2012 15:49:57 +0000
Message-ID: <3DE1F133-2AB9-474E-9C9B-828A78AF4E1D@netflix.com>
References: 
 <CAGJmZPwwZBb7K1GTWiek1PYBfxjp+r=1EsNyHpfX9WSWdCNULw@mail.gmail.com>
In-Reply-To: 
 <CAGJmZPwwZBb7K1GTWiek1PYBfxjp+r=1EsNyHpfX9WSWdCNULw@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
Content-Type: text/plain; charset="iso-8859-1"
Content-ID: <363DD11A60AB094CB611A6666FF804D6@netflix.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0

How much data do you have and how long is "a while"?  In my experience repa=
irs can take a very long time.  Check to see if validation compactions are =
running (nodetool compactionstats) or if files are streaming (nodetool nets=
tats).  If either of those are in progress then your repair should be runni=
ng.  I've seen 12 node, 50G clusters take days to repair to a new data cent=
er.

Not sure if 1.0 is different but in 0.X I don't believe killing the nodetoo=
l process stops the repair.  When we need to stop a repair we have bounced =
all of the participating nodes.  I've been told that there is no harm in st=
opping repairs.

On Apr 24, 2012, at 2:55 PM, Bill Au wrote:

> I am running 1.0.8.  I am adding a new data center to an existing cluster=
.  Following steps outlined in another thread on the mailing list, things w=
ent fine except for the last step, which is to run repair on all the nodes =
in the new data center.  Repair seems to be hanging indefinitely.  There is=
 no activity in system.log.  I did notice that the node being repair is req=
uesting ranges from nodes in both the existing and new data center.  Since =
there is not data in the new data center initially, I though that it may be=
 why repair is hanging.  So I break out of the repair with a control-C afte=
r waiting for a while.  I do see data being added to the new nodes.  When I=
 ran repair for the second time it is still hanging.
>=20
> Why is repair hanging?  Is it save to use control-C to break out of it.  =
How do I recover from this?
>=20
> Bill