Mailing-List: contact jcs-users-help@jakarta.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "JCS Users List" <jcs-users@jakarta.apache.org>
Received-SPF: pass (athena.apache.org: local policy)
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws;
  s=s1024; d=yahoo.com;
  h=X-YMail-OSG:Received:X-Mailer:Date:From:Reply-To:Subject:To:In-Reply-To:MIME-Version:Content-Type:Message-ID;
  b=PLi/yiHnbbqh2zxxAB/w9k/yX91BHClnXG5sHVgDVczoHVvvDiFoyTuY3UEhKh8GvhshafSjzagM8rai+9Kl6euaZko/HyteStI6LpGFDTASZ4P4apWwYcJZYs80OqsPX5Pj2EIbG0hG3Ih6ImlBOjpTebyea3EHbTWIvWy3CYs=;
Date: Thu, 4 Sep 2008 12:22:14 -0700 (PDT)
From: Aaron Smuts <asmuts@yahoo.com>
Reply-To: asmuts@yahoo.com
Subject: Re: JCS using lots of mem for objects written to disk?
To: JCS Users List <jcs-users@jakarta.apache.org>
In-Reply-To: <815517.36202.qm@web33105.mail.mud.yahoo.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Message-ID: <480307.92734.qm@web38704.mail.mud.yahoo.com>

This is great data.  Thanks.  I'll take a look and see if we might be able to trim it down a bit more.

Aaron


--- On Wed, 9/3/08, Nick <nlai@yahoo.com> wrote:

> From: Nick <nlai@yahoo.com>
> Subject: Re: JCS using lots of mem for objects written to disk?
> To: "JCS Users List" <jcs-users@jakarta.apache.org>
> Date: Wednesday, September 3, 2008, 8:09 PM
> Aaron,
> 
> Thanks for your prompt response!
> 
> Here is the results after setting
> jcs.default.cacheattributes.MaxObjects=0,
> and putting two more 3 sec sleeps followed by GCs after the
> cache is loaded:
> 
> 100K objects in the disk cache:
> 
>     TEST STARTING heap size: 2787 KB
>     Purgatory Size = 89171
>     TEST: Sleeping 15 sec to let objs roll to disk
>     Purgatory Size = 0
>     TEST after cache load: 37772 KB, delta 34985 KB
>     TEST after 3 sec pause: 37772 KB, delta 34985 KB
>     TEST after 3 another sec pause: 20216 KB, delta 17429
> KB
>     Purgatory Size = 0
> 
> 200K objects in the disk cache:
> 
>     TEST STARTING heap size: 2787 KB
>     Purgatory Size = 173664
>     TEST: Sleeping 15 sec to let objs roll to disk
>     Purgatory Size = 0
>     TEST after cache load: 57444 KB, delta 54657 KB
>     TEST after 3 sec pause: 57443 KB, delta 54656 KB
>     TEST after 3 another sec pause: 39886 KB, delta 37099
> KB
> 
> Storing no objects in cache reduced the heap, as we'd
> expect.  However,
> we were still holding 190 bytes per item in the disk cache
> (e.g., in the
> 200K objects case, the delta from the start was 37099 KB,
> divided by 200K
> objects is 190 bytes.  For the 100K objects case, it comes
> out 178 bytes.
> 
> I modified my test program to store the same data
> (TestData) in a simple
> HashMap, which resulted in this:
> 
> 100K objects in-mem in HashMap: 
> 
>     TEST STARTING heap size: 2787 KB
>     TEST after cache load: 84312 KB, delta 81525 KB
>     TEST after 3 sec pause: 83903 KB, delta 81116 KB
>     TEST after 3 another sec pause: 83903 KB, delta 81116
> KB
> 
> 200K objects in-mem in HashMap: 
> 
>     TEST STARTING heap size: 2787 KB
>     TEST after cache load: 172580 KB, delta 169793 KB
>     TEST after 3 sec pause: 171600 KB, delta 168813 KB
>     TEST after 3 another sec pause: 171600 KB, delta 168813
> KB
> 
> Note that in the case of using JCS, after loading 200K
> objects all into the
> disk cache, JCS reported Data File Length = 140755580.  So
> of the 168 MB
> being used by the HashMap test, we should expect about 140
> MB of that to be
> data (assuming Java serialization is pretty efficient),
> leaving about 30 MB
> for the HashMap infrastructure.
> 
> I then modified the HashMap test case to store a simple
> "new Object()" in the
> HashMap instead of a TestData.
> 
>     TEST STARTING heap size: 2787 KB
>     TEST after cache load: 16573 KB, delta 13786 KB
>     TEST after 3 sec pause: 16572 KB, delta 13785 KB
>     TEST after 3 another sec pause: 16572 KB, delta 13785
> KB
> 
>     TEST STARTING heap size: 2787 KB
>     TEST after cache load: 33136 KB, delta 30349 KB
>     TEST after 3 sec pause: 33070 KB, delta 30283 KB
>     TEST after 3 another sec pause: 33070 KB, delta 30283
> KB
> 
> So JCS's overhead is really not that bad, compared to a
> HashMap of
> just the keys to Object (about 22% more space).
> 
> I also ran the 200K TestData JCS cache with 0 objects in
> mem case under 
> the YourKit profiler.  It showed 37 MB being used by the
> IndexedDiskCache,
> 36.1 MB of that in java.util.HashMap.  It reported these as
> the primary
> mem users:
> 
> +---------------------------------------------------------------------------+-----------------+-------------------+
> |                                Name                      
>                 |     Objects     |   Shallow Size    |
> +---------------------------------------------------------------------------+-----------------+-------------------+
> |java.lang.String                                          
>                 |  200,429  25 %  |  8,017,160  21 %  |
> |char[]                                                    
>                 |  200,347  25 %  |  7,945,024  21 %  |
> |java.util.HashMap$Entry                                   
>                 |  200,254  25 %  |  9,612,192  26 %  |
> |org.apache.jcs.auxiliary.disk.indexed.IndexedDiskElementDescriptor
>         |  200,000  25 %  |  6,400,000  17 %  |
> 
> The interesting there is that my average String size is 40
> bytes, ditto the
> char[].  Note that I form the key, and the values in
> TestData, using 
> Integer.toString(int).  I guess I'll have to look at
> the code for it, maybe
> its really inefficent.
> 
> After seeing this result, I went back and changed my code
> to use
> "new Integer(i)" as the key, instead of
> "Integer.toString(i)".  Results:
> 
> 100K objects in the disk cache:
> 
>     TEST STARTING heap size: 2787 KB
>     Purgatory Size = 89646
>     TEST: Sleeping 15 sec to let objs roll to disk
>     Purgatory Size = 0
>     TEST after cache load: 32268 KB, delta 29481 KB
>     TEST after 3 sec pause: 14712 KB, delta 11925 KB
>     TEST after 3 another sec pause: 14712 KB, delta 11925
> KB
> 
> 200K objects in the disk cache:
> 
>     TEST STARTING heap size: 2787 KB
>     Purgatory Size = 176268
>     TEST: Sleeping 15 sec to let objs roll to disk
>     Purgatory Size = 0
>     TEST after cache load: 46339 KB, delta 43552 KB
>     TEST after 3 sec pause: 28783 KB, delta 25996 KB
>     TEST after 3 another sec pause: 28783 KB, delta 25996
> KB
>     TEST after 30 another sec pause: 28783 KB, delta 25996
> KB
> 
> This gets us down to a per-disk-cached-item overhead of 130
> bytes.  This may
> be the minimum that overhead achievable with the current
> JCS
> implementation of the indexed disk cache.  
> 
> Nick
> 
> 
> 
>       
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> jcs-users-unsubscribe@jakarta.apache.org
> For additional commands, e-mail:
> jcs-users-help@jakarta.apache.org

---------------------------------------------------------------------
To unsubscribe, e-mail: jcs-users-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jcs-users-help@jakarta.apache.org