Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@couchdb.apache.org
MIME-Version: 1.0
From: DA <daniel.adam903@gmail.com>
Date: Wed, 30 Nov 2016 19:50:30 +0100
Message-ID: <CAMw+16j_yhwNcqtuOuWpsQ9Wg1KjeK65ctK0AFQ4OuGf_HhzSg@mail.gmail.com>
Subject: Re: view/index rebuild in v2.0
To: user@couchdb.apache.org
Content-Type: multipart/alternative; boundary=f403045db942da3c2f0542892d84
archived-at: Wed, 30 Nov 2016 18:50:36 -0000

--f403045db942da3c2f0542892d84
Content-Type: text/plain; charset=UTF-8

Hi,
thanks Peter, but I was thinking of a measures suitable for production usage.

Meanwhile, few observations:
- the 8 respective indexer tasks are not progressing evenly, some are
at 50%, some at 90%
- I noticed the system usage fell by almost 50%, both CPU and disk activity.
So the system is now even more under-utilized, quite strange.

When it's completed, I will try the same excercise on an SSD disk.

Cheers, Daniel


On Wed, Nov 30, 2016 at 10:03 AM Peyton Vaughn <pvaughn@6fusion.com> wrote:

> Hi Daniel,
> Do you have 'delayed_commits' set to true? It's now defaulted to false in
> couch 2.0, but it really causes a pretty hefty performance hit when
> disabled.
>
> Peyton
>
> On Wed, Nov 30, 2016 at 5:57 AM, Daniel Adam <daniel.adam903@gmail.com>
> wrote:
>
> > Hi,
> > I'm running an index/view re-generation with 10M documents, 10views,
> having
> > 1 node and 8 shards on it.
> > It seems it will take more than 1 day. What are limiting factors for
> that?
> > Looking at perfromance monitoring tools (Task Manager and Process
> > Explorer), it seems neither I/O or CPU is saturated:
> > CPU (8 cores) each core taking <30% its capacity
> > I/O reads ~ 8MB/s
> > I/O writes ~ 1.5MB/s
> >
> > Windows 7 64bit
> > 16GB memory
> > Intel i7-4800 @ 2.7GHz
> > HDD Seagate ST500LM0
> >
> > Any explanation why the system limits are not being saturated? Seems to
> me
> > the bottleneck is I/O, but the disk is capable of 100MB/s throughput in
> > sequential reads/writes. If the limiting factor is indeed I/O, I assume
> > changing number of shards would not gain much, possibly even 1 shard
> would
> > provide a similar speed.
> >
> > Any way to improve the speed of index rebuild?
> >
> > Thanks,
> > Daniel
> >
>

--f403045db942da3c2f0542892d84--