couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Randall Leeds <randall.le...@gmail.com>
Subject Re: few doubts
Date Mon, 04 Jul 2011 22:01:54 GMT
On Mon, Jul 4, 2011 at 14:13, Robert Newson <rnewson@apache.org> wrote:
> From (vague) memory, I thought page cache was released when you close
> the file descriptor.
>
> B.

I known I've subjectively felt like this was the case based on performance.
Restarting CouchDB on a running box I don't notice any change in /proc/meminfo.
I couldn't find anything related to this in:
http://www.kernel.org/doc/Documentation/sysctl/vm.txt
It'd be nice to know.

>
> On 4 July 2011 21:55, Randall Leeds <randall.leeds@gmail.com> wrote:
>> On Mon, Jul 4, 2011 at 13:34, Randall Leeds <randall.leeds@gmail.com> wrote:
>>>
>>> On Jul 4, 2011 11:11 AM, "Jens Alfke" <jens@mooseyard.com> wrote:
>>>>
>>>>
>>>> On Jul 3, 2011, at 11:41 PM, Matthew John wrote:
>>>>
>>>> > how does CouchDB perform
>>>> > when compared to MySql in performing the Warm-up (lets define Warm-up
as
>>>> > filling the memory with Disk data in a judicial manner).
>>>>
>>>> No idea. But really, you could perform something very similar by doing
>>>> something like
>>>>        cat /path/to/my_database.couch > /dev/null
>>>> since that would have the same effect of loading the file contents into
>>>> the VM cache. So it’s not really that specific to CouchDB itself.
>>>>
>>>> How large is your data-set? How many rows, and what’s the disk space
>>>> occupied by the MySQL tables?
>>>>
>>>> —Jens
>>>
>>> It should also be noted that CouchDB does not do any caching of its own. I
>>> don't know if MySQL does, but if yes then your warm-up query is possibly
>>> letting MySQL cache some index headers or something, but nothing exactly
>>> like this happens in CouchDB. The only caching is at the filesystem/OS
>>> level. In fact, if your OS hasn't decided to dump the page cache, it may be
>>> beneficial _not_ to do anything that might disturb it between restarts,
>>> since some of that file data might still be around. This behavior would
>>> depend on your OS keeping the pages around after all file descriptors are
>>> closed. I don't know whether that happens or when. Maybe there is a kernel
>>> setting for this behavior?
>>
>> Ah. Of course. The kernel parameter is /sys/proc/vm/swappiness on Linux systems.
>> 1 - Dump cached pages in favor of keeping applications in memory
>> ...
>> ...
>> 100 - Swap applications in favor of keeping pages cached in memory
>>
>> I suspect there's no aggressive dumping of pages just because files
>> have been closed, though I haven't found any information one way or
>> another yet. In other words, while your warm-up query might do things
>> for MySQL's internal caching that could have benefits, doing something
>> similar in CouchDB would probably just force out whatever useful pages
>> your OS has cached from before the restart. I would not recommend
>> trying to do anything like this.
>>
>

Mime
View raw message