incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brian Candler <B.Cand...@pobox.com>
Subject Re: Timeout Error when trying to access views + Indexing problems
Date Sun, 04 Oct 2009 20:49:33 GMT
On Sun, Oct 04, 2009 at 11:29:29AM -0700, Glenn Rempe wrote:
>    Thanks for the reply Brian. ? This particular part of the issue (slow
>    bulk post of new records) was resolved earlier this morning with a
>    suggestion from Chris Anderson in this thread who suggested the
>    following ini change which I made in futon:
> 
>    [uuids]
>    algorithm = sequential
> 
>    This totally resolved the slow bulk updates for me (instantly!).

OK, cool.

I've been generating sequential uuids client-side for a long time for other
reasons (basically so that items with equal keys in a view are shown in
insertion order)

However I'm surprised it makes a factor of 30 difference. I've done
benchmarking tests with inserting 1000 documents at a time, where I didn't
provide uuids client-side, and performance was just fine.

>    on a quad core 15GB RAM EC2 X-Large server storing the data on
>    an EBS volume. ? Huge difference.

Ah. Possibly EBS is the key factor here. With local disk it's cheap to have
a lot of round trips to the local VFS cache (cheap), followed by single
flush to disk. But perhaps EBS bypasses the VFS cache? That is, maybe each
block is being sent and received over the wire multiple times, e.g. if it's
being read, updated and rewritten repeatedly?

A tcpdump might shed some light on this.

Regards,

Brian.

Mime
View raw message