incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Anderson <jch...@apache.org>
Subject Re: Progress of view generation
Date Thu, 22 Jan 2009 19:22:18 GMT
On Thu, Jan 22, 2009 at 11:18 AM, Chris Wagner
<christopher.t.wagner@gmail.com> wrote:
> FYI, this ended up taking a matter of days.  I believe I was generating
> two views at once, though.  I'm uncertain whether a third view that I
> have (in the same design document) was also being generated, however.
> My view file is approximately 1.25 times the size of the database file.
>

all views in a design doc are generated in a single batch. This makes
for less IO overhead to the Javascript process, as well as bulk
operations to couchdb's on disk btrees.

> Thanks for your help, Paul. :)
>
>
> On Wed, 2009-01-14 at 16:58 +0000, Paul Carey wrote:
>> Hi Chris
>>
>> > I have a database containing more than 11-million documents (87 GB).
>> >
>> > As can be expected, the view generation is taking quite a long time.
>> > The problem is, though, that I have no idea how long it could take; I'm
>> > not even sure whether it could be a matter of hours, days, weeks, ...
>>
>> At a guess, hours, probably quite a few.
>>
>> > So, I guess I am wondering if there's any way to gauge the progress of
>> > the generation of these views -- or at least to be sure the views are
>> > actually being generated and I'm not just watching the Futon
>> > spinny-thing go on endlessly.
>>
>> There is indeed. Take a look at
>>
>> /usr/local/var/lib/couchdb/.my_design_doc.couch/Myview.view
>>
>> You should be able to see it growing while view generation is taking
>> place.  Given that you're emitting the doc as a value, and assuming
>> your map function as is you stated e.g. no filtering is being
>> performed, you can expect the size of the view file to be in the same
>> ballpark as the size of you database.
>>
>> > That is, once the view has been generated once, can I be confident
>> > that further generations (assuming I've only added a small number of
>> > documents since the last generation) will be reasonably quick?
>>
>> Yes.
>>
>> Paul
>
>



-- 
Chris Anderson
http://jchris.mfdz.com

Mime
View raw message