Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@couchdb.apache.org
Received-SPF: pass (athena.apache.org: domain of
 christopher.t.wagner@gmail.com designates 209.85.198.228 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=subject:from:to:in-reply-to:references:content-type:date:message-id
         :mime-version:x-mailer:content-transfer-encoding;
        b=xjufwDmjiijb5m4Ef5p0tCvdeSeIhYre2uOIn5s+vOwcYlYVQ6J+CcDd68Uc+b2jWl
         f0a6ShQLXVPdSKQrHwjysyrQNh90fbmvaG6Zzl0scthXDq7jxwg+jwmTrMeES8lv4iNB
         OEdnBEcIwAx+1XzF8nUSvWmyd7Ab5lh+knbJE=
Subject: Re: Progress of view generation
From: Chris Wagner <christopher.t.wagner@gmail.com>
To: user@couchdb.apache.org
In-Reply-To: <13ae4dd50901140858x5e5c450uf14b67698756af3e@mail.gmail.com>
References: <1231947933.543.15.camel@chris-laptop>
	 <13ae4dd50901140858x5e5c450uf14b67698756af3e@mail.gmail.com>
Content-Type: text/plain
Date: Thu, 22 Jan 2009 14:18:31 -0500
Message-Id: <1232651911.13437.64.camel@chris-laptop>
Mime-Version: 1.0
Content-Transfer-Encoding: 7bit

FYI, this ended up taking a matter of days.  I believe I was generating
two views at once, though.  I'm uncertain whether a third view that I
have (in the same design document) was also being generated, however.
My view file is approximately 1.25 times the size of the database file.

Thanks for your help, Paul. :)


On Wed, 2009-01-14 at 16:58 +0000, Paul Carey wrote:
> Hi Chris
> 
> > I have a database containing more than 11-million documents (87 GB).
> >
> > As can be expected, the view generation is taking quite a long time.
> > The problem is, though, that I have no idea how long it could take; I'm
> > not even sure whether it could be a matter of hours, days, weeks, ...
> 
> At a guess, hours, probably quite a few.
> 
> > So, I guess I am wondering if there's any way to gauge the progress of
> > the generation of these views -- or at least to be sure the views are
> > actually being generated and I'm not just watching the Futon
> > spinny-thing go on endlessly.
> 
> There is indeed. Take a look at
> 
> /usr/local/var/lib/couchdb/.my_design_doc.couch/Myview.view
> 
> You should be able to see it growing while view generation is taking
> place.  Given that you're emitting the doc as a value, and assuming
> your map function as is you stated e.g. no filtering is being
> performed, you can expect the size of the view file to be in the same
> ballpark as the size of you database.
> 
> > That is, once the view has been generated once, can I be confident
> > that further generations (assuming I've only added a small number of
> > documents since the last generation) will be reasonably quick?
> 
> Yes.
> 
> Paul