lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shawn Heisey <s...@elyograg.org>
Subject Re: Question about cache
Date Fri, 11 May 2012 18:25:14 GMT
On 5/11/2012 9:30 AM, Anderson vasconcelos wrote:
> HI  Kuli
>
> The free -m command gives me
>                     total       used       free     shared    buffers
> cached
> Mem:          9991       9934         57          0         75       5759
> -/+ buffers/cache:       4099       5892
> Swap:         8189       3395       4793
>
> You can see that has only 57m free and 5GB cached.
>
> In top command, the glassfish process used 79,7% of memory:
>
>   PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+
> COMMAND
>   4336 root      21   0 29.7g 7.8g 4.0g S         0.3      79.7   5349:14
> java
>
>
> If i increase the memory of server for more 2GB, the SO will be use this
> additional 2GB in cache? I need to increse the memory size?

Are you having a problem you need to track down, or are you just raising 
a concern because your memory usage is not what you expected?

It is 100% normal for a Linux system to show only a few megabytes of 
memory free.  To make things run faster, the OS caches disk data using 
memory that is not directly allocated to programs or the OS itself.  If 
a program requests memory, the OS will allocate it immediately, it 
simply forgets the least used part of the cache.

Windows does this too, but Microsoft decided that novice users would 
freak out if the task manager were to give users the true picture of 
memory usage, so they exclude disk cache when calculating free memory.  
It's not really a lie, just not the full true picture.

A recent version of Solr (3.5, if I remember right) made a major change 
in the way that the index files are accessed.  The way things are done 
now is almost always faster, but it makes the memory usage in the top 
command completely useless.  The VIRT memory size includes all of your 
index files, plus all the memory that the java process is capable of 
allocating, plus a little that i can't quite account for.  The RES size 
is also bigger than expected, and I'm not sure why.

Based on the numbers above, I am guessing that your indexes take up 
15-20GB of disk space.  For best performance, you would want a machine 
with at least 24GB of RAM so that your entire index can fit into the OS 
disk cache.  The 10GB you have (which leaves the 5.8 GB for disk cache 
as you have seen) may be good enough to cache the frequently accessed 
portions of your index, so your performance might be just fine.

Thanks,
Shawn


Mime
View raw message