jakarta-jcs-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vladimir Olenin" <VOle...@cihi.ca>
Subject RE: configuration question
Date Mon, 28 Aug 2006 21:23:07 GMT
Still having problems with 'Purgatory' settings...

Questions:

1) Is the purgatory object pool ever being 'shrinked' (when all data is
already written to disk)
2) if so, does JCS do it automatically or there are some specific
settings? (couldn't find anything in the docs)
3) Is there a way to 'ask' Purgatory to 'block' any subsequent 'put'
requests until the current 'bounded buffer' (with MaxPurgatorySize) is
dumped to the disk and space in the purgatory becomes available again (I
realized it's quite against the rational why the Purgotory was
introduced in the first place, but read on)


The thing I need to do is this:

1) I need to put, lets say, 100K entries into the cache, and keep _ONLY_
1K of them readily available in the memory. The rest should be
dynamically load from the disk when required (replacing least frequently
used entries in memory)
2) the cache is written into only ONCE, at the application startup
3) the cache is then accessed (read-only) extensively during the
application life cycle


The problem I'm having is this:

1) with current settings (see below) everything works as expected, with
only one exception:
2) the memory consumption with 'unbounded' Purgatory size
(MaxPurgatorySize=-1, correct?) is at least 150M higher than a bounded
one (the values I tried ranged from 100 to 1000), even after all data is
loaded into cache. In my case 100M higher means twice as much (300M vs
150M)
3) while it's unknown right now whether the memory would eventually be
freed-up, I suspect it might not
4) with bounded purgatory (MaxPurgatorySize = 1000), some entries got
evicted from the cache before they were written to the disk.


Below are my current settings:

#----------------------
jcs.default = DC_SMALL_FOOTPRINT
jcs.default.cacheattributes =
org.apache.jcs.engine.CompositeCacheAttributes
jcs.default.cacheattributes.MaxObjects = 1000
jcs.default.cacheattributes.MemoryCacheName =
org.apache.jcs.engine.memory.lru.LRUMemoryCache
jcs.default.cacheattributes.DiskUsagePatternName = UPDATE

jcs.auxiliary.DC_SMALL_FOOTPRINT=org.apache.jcs.auxiliary.disk.indexed.I
ndexedDiskCacheFactory
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes=org.apache.jcs.auxiliary.dis
k.indexed.IndexedDiskCacheAttributes
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes.maxKeySize=-1
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes.MaxPurgatorySize=-1
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes.DiskPath=jcs
#----------------------

Thanks!

-----Original Message-----
From: Aaron Smuts [mailto:asmuts@yahoo.com] 
Sent: Monday, August 28, 2006 9:33 AM
To: JCS Users List
Subject: RE: configuration question



--- Vladimir Olenin <VOlenin@cihi.ca> wrote:

> Thanks. It seems to work now. I wonder if 'UPDATE'
> policy is set, does
> 'MaxPurgatorySize' property has any effect then?

Yes.  In JCS mythology, purgatory is simply a queue on the way to disk,
where the items can be accessed. 
It's a map on top of a queue.  All items that go to disk, go to the
queue.  You can set a limit on the queue size regardless of the disk
usage pattern.



> What's the lifecycle of
> cache element with UPDATE policy? Does it go just between 'memory' and

> 'dc'? (if not found in memory, load from disk and put in memory cache,

> expiring some entries from memory on the way if necessary)?

When the item is put into the cache for the first time it goes to disk
as well as memory (if the max memory size is greater than 0).  If the
item is later now in memory and is retrieved from disk, it will go back
into memory, but it should will still be on disk.  If the item is
evicted from memory, because, let's say, it falls off the LRU, then it
will NOT be put to disk
again.  This prevents thrashing.   . . .  

(I want to make another pattern called PRIMARY where the items just go
to disk until they are accessed yet again . . .)

> 
> Thanks for quick answers, Aaron.
> 
> Vlad
> 
> -----Original Message-----
> From: Aaron Smuts [mailto:asmuts@yahoo.com]
> Sent: Monday, August 28, 2006 9:21 AM
> To: JCS Users List
> Subject: RE: configuration question
> 
> 
> 
> --- Vladimir Olenin <VOlenin@cihi.ca> wrote:
> Sorry, the docs are wrong.  DiskUsagePattern is the property but you 
> have to set it with a "name".  Use "DiskUsagePatternName=UPDATE"
> 
> 
> > Trying to do this as per docs, with line
> > 
> > jcs.default.cacheattributes.DiskUsagePattern =
> UPDATE
> > 
> > But getting this in the output log:
> > 
> > [WARN] Failed to set property diskUsagePattern to
> value "UPDATE".
> > Conversion to type [short] failed.
> > 
> > 
> > I think 'UPDATE' is a final short variable
> somewhere in the code.... 
> > But there is no referral to it in the guides. What
> should it be?
> > 
> > -----Original Message-----
> > From: Aaron Smuts [mailto:asmuts@yahoo.com]
> > Sent: Friday, August 25, 2006 8:21 PM
> > To: JCS Users List
> > Subject: Re: configuration question
> > 
> > Purgatory is a queue on which items on the way to
> disk can still be 
> > accessed.  Setting a limit on purgatory allows you
> to prevent the 
> > queue from getting too large.  If you set a
> purgatory to 1000 and then
> 
> > quickly add 100,000 items faster than they can be
> written to disk, 
> > then the queue will certainly hit 1000.  When the
> max purgatory size 
> > is reached the oldest are dropped.
> > 
> > Purgatory is basically a bounded queue with a map
> on top.  There is a 
> > bit more description here:
> > 
> >
>
http://jakarta.apache.org/jcs/IndexedDiskAuxCache.html
> > 
> > You should probably run with the cacheattributes
> properly called 
> > DiskUsagePatternName=UPDATE.  This will cause the
> items to go to disk 
> > when they are added to the cache instead of when
> they fall out of the 
> > Memory cache.
> > 
> > If you are going to flood the cache with items,
> then don't set a 
> > purgatory limit, or set it to the number you will
> be adding if it is 
> > known.
> > 
> > Aaron
> > 
> > --- Vladimir Olenin <VOlenin@cihi.ca> wrote:
> > 
> > > Hi.
> > >  
> > > I need to configure JCS in the following manner:
> > >  
> > > - need to load 100K entries into the cache
> > > - NONE of the entries are allowed to be expired
> > (ie, ALL must be
> > > eternal)
> > > - have to keep in memory not more than 1000
> > entries at a time
> > > - the rest entries should be swapped to disk
> > automatically
> > >  
> > > What is the default setting that would work the
> > best? The current
> > > configuration I use is below and it seem to
> work.
> > I can't quite figure
> > 
> > > out the meaning of 'MaxPurgatorySize' property.
> > From the docs I
> > > understood that it's just the size of the 'disk
> > purge' queue. But the
> > > tests show that if it's a positive number, then
> > the entries start to
> > > be 'expired' from the cache (eg, if set to 1000
> it
> > seems like only a
> > > small fraction of 100K entries remains in cache
> > and some of the
> > > entries put in the cache become missing for
> sure,
> > since jcs.get
> > > returns null).
> > >  
> > > Thanks!.
> > >  
> > > ===========================================
> > > jcs.default = DC_SMALL_FOOTPRINT
> > > jcs.default.cacheattributes =
> > > org.apache.jcs.engine.CompositeCacheAttributes
> > > jcs.default.cacheattributes.MaxObjects = 1000 
> > > jcs.default.cacheattributes.MemoryCacheName = 
> > > org.apache.jcs.engine.memory.lru.LRUMemoryCache
> > >  
> > >
> >
>
jcs.auxiliary.DC_SMALL_FOOTPRINT=org.apache.jcs.auxiliary.disk.indexed.I
> > > ndexedDiskCacheFactory
> > >
> >
>
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes=org.apache.jcs.auxiliary.dis
> > > k.indexed.IndexedDiskCacheAttributes
> > >
> >
>
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes.DiskPath=c:/Projects/Grouper
> > > /temp/jcs
> > >
> >
>
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes.maxKeySize=-1
> > >
> >
>
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes.MaxPurgatorySize=-1
> > > 
> > 
> > 
> >
>
---------------------------------------------------------------------
> > To unsubscribe, e-mail:
> > jcs-users-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail:
> > jcs-users-help@jakarta.apache.org
> > 
> > 
> >
>
---------------------------------------------------------------------
> > To unsubscribe, e-mail:
> > jcs-users-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail:
> > jcs-users-help@jakarta.apache.org
> > 
> > 
> 
> 
>
---------------------------------------------------------------------
> To unsubscribe, e-mail:
> jcs-users-unsubscribe@jakarta.apache.org
> For additional commands, e-mail:
> jcs-users-help@jakarta.apache.org
> 
> 
>
---------------------------------------------------------------------
> To unsubscribe, e-mail:
> jcs-users-unsubscribe@jakarta.apache.org
> For additional commands, e-mail:
> jcs-users-help@jakarta.apache.org
> 
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: jcs-users-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jcs-users-help@jakarta.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: jcs-users-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jcs-users-help@jakarta.apache.org


Mime
View raw message