couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From DA <daniel.adam...@gmail.com>
Subject Re: view/index rebuild in v2.0
Date Wed, 30 Nov 2016 18:50:30 GMT
Hi,
thanks Peter, but I was thinking of a measures suitable for production usage.

Meanwhile, few observations:
- the 8 respective indexer tasks are not progressing evenly, some are
at 50%, some at 90%
- I noticed the system usage fell by almost 50%, both CPU and disk activity.
So the system is now even more under-utilized, quite strange.

When it's completed, I will try the same excercise on an SSD disk.

Cheers, Daniel


On Wed, Nov 30, 2016 at 10:03 AM Peyton Vaughn <pvaughn@6fusion.com> wrote:

> Hi Daniel,
> Do you have 'delayed_commits' set to true? It's now defaulted to false in
> couch 2.0, but it really causes a pretty hefty performance hit when
> disabled.
>
> Peyton
>
> On Wed, Nov 30, 2016 at 5:57 AM, Daniel Adam <daniel.adam903@gmail.com>
> wrote:
>
> > Hi,
> > I'm running an index/view re-generation with 10M documents, 10views,
> having
> > 1 node and 8 shards on it.
> > It seems it will take more than 1 day. What are limiting factors for
> that?
> > Looking at perfromance monitoring tools (Task Manager and Process
> > Explorer), it seems neither I/O or CPU is saturated:
> > CPU (8 cores) each core taking <30% its capacity
> > I/O reads ~ 8MB/s
> > I/O writes ~ 1.5MB/s
> >
> > Windows 7 64bit
> > 16GB memory
> > Intel i7-4800 @ 2.7GHz
> > HDD Seagate ST500LM0
> >
> > Any explanation why the system limits are not being saturated? Seems to
> me
> > the bottleneck is I/O, but the disk is capable of 100MB/s throughput in
> > sequential reads/writes. If the limiting factor is indeed I/O, I assume
> > changing number of shards would not gain much, possibly even 1 shard
> would
> > provide a similar speed.
> >
> > Any way to improve the speed of index rebuild?
> >
> > Thanks,
> > Daniel
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message