couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peyton Vaughn <pvau...@6fusion.com>
Subject Re: view/index rebuild in v2.0
Date Wed, 30 Nov 2016 19:08:47 GMT
I'm not sure why that wouldn't work for production usage. Commencing a
shutdown on receipt of SIGTERM/SIGINT is pretty standard practice -
mongodb, postgres, mysql, oracle - all do this, as examples.

On Wed, Nov 30, 2016 at 1:50 PM, DA <daniel.adam903@gmail.com> wrote:

> Hi,
> thanks Peter, but I was thinking of a measures suitable for production
> usage.
>
> Meanwhile, few observations:
> - the 8 respective indexer tasks are not progressing evenly, some are
> at 50%, some at 90%
> - I noticed the system usage fell by almost 50%, both CPU and disk
> activity.
> So the system is now even more under-utilized, quite strange.
>
> When it's completed, I will try the same excercise on an SSD disk.
>
> Cheers, Daniel
>
>
> On Wed, Nov 30, 2016 at 10:03 AM Peyton Vaughn <pvaughn@6fusion.com>
> wrote:
>
> > Hi Daniel,
> > Do you have 'delayed_commits' set to true? It's now defaulted to false in
> > couch 2.0, but it really causes a pretty hefty performance hit when
> > disabled.
> >
> > Peyton
> >
> > On Wed, Nov 30, 2016 at 5:57 AM, Daniel Adam <daniel.adam903@gmail.com>
> > wrote:
> >
> > > Hi,
> > > I'm running an index/view re-generation with 10M documents, 10views,
> > having
> > > 1 node and 8 shards on it.
> > > It seems it will take more than 1 day. What are limiting factors for
> > that?
> > > Looking at perfromance monitoring tools (Task Manager and Process
> > > Explorer), it seems neither I/O or CPU is saturated:
> > > CPU (8 cores) each core taking <30% its capacity
> > > I/O reads ~ 8MB/s
> > > I/O writes ~ 1.5MB/s
> > >
> > > Windows 7 64bit
> > > 16GB memory
> > > Intel i7-4800 @ 2.7GHz
> > > HDD Seagate ST500LM0
> > >
> > > Any explanation why the system limits are not being saturated? Seems to
> > me
> > > the bottleneck is I/O, but the disk is capable of 100MB/s throughput in
> > > sequential reads/writes. If the limiting factor is indeed I/O, I assume
> > > changing number of shards would not gain much, possibly even 1 shard
> > would
> > > provide a similar speed.
> > >
> > > Any way to improve the speed of index rebuild?
> > >
> > > Thanks,
> > > Daniel
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message