incubator-couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Filipe David Manana <fdman...@gmail.com>
Subject Re: ways to improve compaction
Date Wed, 26 May 2010 17:47:15 GMT
On Mon, Dec 21, 2009 at 11:20 PM, Damien Katz <damien@apache.org> wrote:

> I saw recently some issues people where having with compaction, and I
> thought I'd get some thoughts down about ways to improve the compaction
> code/experience.
>
>
> 5. Better Low level file driver support. Because we are using the Erlang
> built-in file system drivers, we don't have access to a lot of flags. If we
> had our own drivers, one option we'd like to use is to not OS cache the
> reads and write during the compaction, it's unnecessary for compaction and
> it could completely consume the cache with rarely accessed data, evicting
> lots of recently used live data, greatly hurting performance of other
> databases.
>

Related to this, I got a patch accepted for Erlang/OTP branch dev, which
adds posix_fadvise to the file module:

http://github.com/erlang/otp/commit/be2ebfd0f2b8e15363288d1dab4c177c237e234a

In the long term we can possibly use it for CouchDB (with Erlang R14
something).


>
> Anyway, just getting these thoughts out. More ideas and especially code
> welcome.
>
> -Damien




-- 
Filipe David Manana,
fdmanana@gmail.com

"Reasonable men adapt themselves to the world.
Unreasonable men adapt the world to themselves.
That's why all progress depends on unreasonable men."

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message