Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@couchdb.apache.org
Received-SPF: pass (nike.apache.org: domain of jchris@gmail.com designates
 74.125.46.30 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:sender:in-reply-to:references:date
         :x-google-sender-auth:message-id:subject:from:to:content-type
         :content-transfer-encoding;
        b=tz+FxX+J3QBP6sW7f78SxNYFlgD7yzUd12Vj9qscmdUfjOLon0hX9r1m/t3V4ipr0O
         B3ihCjY8Ra/5z2bMq9V11wle1+5MdvhjCzpla7mwjanmmcjfoLSVcRBbVuecaZ6V2LSw
         qKTzuBy9xR+iYRhG/LwE04BvvfnYk+ScGpdTs=
MIME-Version: 1.0
Sender: jchris@gmail.com
In-Reply-To: <1232651911.13437.64.camel@chris-laptop>
References: <1231947933.543.15.camel@chris-laptop>
	 <13ae4dd50901140858x5e5c450uf14b67698756af3e@mail.gmail.com>
	 <1232651911.13437.64.camel@chris-laptop>
Date: Thu, 22 Jan 2009 11:22:18 -0800
Message-ID: <e282921e0901221122i7cfad12ek4756e12baea8885a@mail.gmail.com>
Subject: Re: Progress of view generation
From: Chris Anderson <jchris@apache.org>
To: user@couchdb.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

On Thu, Jan 22, 2009 at 11:18 AM, Chris Wagner
<christopher.t.wagner@gmail.com> wrote:
> FYI, this ended up taking a matter of days.  I believe I was generating
> two views at once, though.  I'm uncertain whether a third view that I
> have (in the same design document) was also being generated, however.
> My view file is approximately 1.25 times the size of the database file.
>

all views in a design doc are generated in a single batch. This makes
for less IO overhead to the Javascript process, as well as bulk
operations to couchdb's on disk btrees.

> Thanks for your help, Paul. :)
>
>
> On Wed, 2009-01-14 at 16:58 +0000, Paul Carey wrote:
>> Hi Chris
>>
>> > I have a database containing more than 11-million documents (87 GB).
>> >
>> > As can be expected, the view generation is taking quite a long time.
>> > The problem is, though, that I have no idea how long it could take; I'm
>> > not even sure whether it could be a matter of hours, days, weeks, ...
>>
>> At a guess, hours, probably quite a few.
>>
>> > So, I guess I am wondering if there's any way to gauge the progress of
>> > the generation of these views -- or at least to be sure the views are
>> > actually being generated and I'm not just watching the Futon
>> > spinny-thing go on endlessly.
>>
>> There is indeed. Take a look at
>>
>> /usr/local/var/lib/couchdb/.my_design_doc.couch/Myview.view
>>
>> You should be able to see it growing while view generation is taking
>> place.  Given that you're emitting the doc as a value, and assuming
>> your map function as is you stated e.g. no filtering is being
>> performed, you can expect the size of the view file to be in the same
>> ballpark as the size of you database.
>>
>> > That is, once the view has been generated once, can I be confident
>> > that further generations (assuming I've only added a small number of
>> > documents since the last generation) will be reasonably quick?
>>
>> Yes.
>>
>> Paul
>
>


-- 
Chris Anderson
http://jchris.mfdz.com