Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@couchdb.apache.org
Received-SPF: pass (nike.apache.org: domain of fabio.forno@gmail.com
 designates 209.85.210.176 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:date:message-id:subject:from:to:content-type;
        b=qH5hPq4JZuVliBa7dcADpnj9llZIsUtfljM6xtP2er6fdo2N6EGj5r/sdXQBS6M1V6
         qPOdsApv4MZZBGGTIzX/iNZO4AG76BkcjX76PFOCdWF9W3BREPx3GKjVrVzN5h/d87CI
         FSzH+4jPrMq/103kP3ttUvGDNauKaHrXwd/5Q=
MIME-Version: 1.0
Date: Thu, 22 Oct 2009 12:55:19 +0200
Message-ID: <2fd53c3a0910220355i3bcbe880j385cf68e61cd570b@mail.gmail.com>
Subject: re-index efficiency
From: Fabio Forno <fabio.forno@gmail.com>
To: user@couchdb.apache.org
Content-Type: text/plain; charset=ISO-8859-1

Hi,
not knowing the internals of couchdb I may ask stupid question, so
just ignore it if it's really stupid ;)

Using it I've noticed the re-index times take a time which comparable
to the insertion off all the documents without using bulk inserts,
while with bulk inserts the insert ionof documents is much faster.
Instead in my idea, re-indexing should be as fast as fast bulk
inserts, since when computing an index we don't need to do many
fsyncs, but instead allow maximum caching before disk writes (with
berkeley db for example, sustained write of data exceeding the memory
cache are 100-1000x faster without syncs for each write). So, since I
don't think that this relative slowness is due to fsyncs which is the
main reason? (another hint which rules out fsyncs is that cpu is
rather high and not in waiting state)

-- 
Fabio Forno,
Bluendo srl http://www.bluendo.com
jabber id: ff@jabber.bluendo.com