Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: pass (nike.apache.org: domain of jonathan.haddad@gmail.com
 designates 74.125.82.182 as permitted sender)
MIME-Version: 1.0
Sender: jonathan.haddad@gmail.com
In-Reply-To: <eaf243efebde44d29f96f050890cce1e@TX1ITMN105.match.corp>
References: <eaf243efebde44d29f96f050890cce1e@TX1ITMN105.match.corp>
Date: Fri, 26 Sep 2014 10:40:27 -0700
Message-ID: 
 <CACUnPaDgdyFPmzXdC8MpVn1PppiWxuo2cpQk8nmJcWz_moeg=w@mail.gmail.com>
Subject: Re: Repair taking long time
From: Jonathan Haddad <jon@jonhaddad.com>
To: "user@cassandra.apache.org" <user@cassandra.apache.org>
Content-Type: text/plain; charset=UTF-8

Are you using Cassandra 2.0 & vnodes?  If so, repair takes forever.
This problem is addressed in 2.1.

On Fri, Sep 26, 2014 at 9:52 AM, Gene Robichaux
<Gene.Robichaux@match.com> wrote:
> I am fairly new to Cassandra. We have a 9 node cluster, 5 in one DC and 4 in
> another.
>
>
>
> Running a repair on a large column family seems to be moving much slower
> than I expect.
>
>
>
> Looking at nodetool compaction stats it indicates the Validation phase is
> running that the total bytes is 4.5T (4505336278756).
>
>
>
> This is a very large CF. The process has been running for 2.5 hours and has
> processed 71G (71950433062). That rate is about 28.4 GB per hour. At this
> rate it will take 158 hours, just shy of 1 week.
>
>
>
> Is this reasonable? This is my first large repair and I am wondering if this
> is normal for a CF of this size. Seems like a long time to me.
>
>
>
> Is it possible to tune this process to speed it up? Is there something in my
> configuration that could be causing this slow performance? I am running
> HDDs, not SSDs in a JBOD configuration.
>
>
>
>
>
>
>
> Gene Robichaux
>
> Manager, Database Operations
>
> Match.com
>
> 8300 Douglas Avenue I Suite 800 I Dallas, TX  75225
>
> Phone: 214-576-3273
>
>


-- 
Jon Haddad
http://www.rustyrazorblade.com
twitter: rustyrazorblade