jakarta-jcs-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aaron Smuts <asm...@yahoo.com>
Subject RE: configuration question
Date Mon, 28 Aug 2006 22:02:55 GMT
Yes, you should be able to setup a queue that will
block or try to run in the current thread.  Configure
it to use a pooled type of queue.  Set that queue with
a when blocked policy of BLOCK or RUN.  Configure the
purgatory size to -1 or to higher than the max buffer
size of the queue.  This way the queue will
effectively manage the purgatory size.  (This does
make the disk cache potentially blocking.  I've never
tried this but it shoudl work.)

    /** abort when queue is full and max threads is
reached. */
    public static final String POLICY_ABORT = "ABORT";

    /** block when queue is full and max threads is
reached. */
    public static final String POLICY_BLOCK = "BLOCK";

    /** run in current thread when queue is full and
max threads is reached. */
    public static final String POLICY_RUN = "RUN";

    /** wait when queue is full and max threads is
reached. */
    public static final String POLICY_WAIT = "WAIT";

    /** discard oldest when queue is full and max
threads is reached. */
    public static final String POLICY_DISCARDOLDEST =
"DISCARDOLDEST";

There are some details a the bottom of this page:
http://jakarta.apache.org/jcs/IndexedDiskAuxCache.html

Also, you can call getStats to see how many items are
in purgatory etc.


[YOUR FILE]
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes.EventQueueType=POOLED
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes.EventQueuePoolName=disk_cache_event_queue

##############################################################
################## OPTIONAL THREAD POOL CONFIGURATION
########

# Disk Cache Event Queue Pool
thread_pool.disk_cache_event_queue.useBoundary=true
thread_pool.disk_cache_event_queue.boundarySize=50000
thread_pool.disk_cache_event_queue.maximumPoolSize=15
thread_pool.disk_cache_event_queue.minimumPoolSize=1
thread_pool.disk_cache_event_queue.keepAliveTime=3500
thread_pool.disk_cache_event_queue.startUpSize=1
thread_pool.disk_cache_event_queue.whenBlockedPolicy=RUN



--- Vladimir Olenin <VOlenin@cihi.ca> wrote:

> Still having problems with 'Purgatory' settings...
> 
> Questions:
> 
> 1) Is the purgatory object pool ever being
> 'shrinked' (when all data is
> already written to disk)
> 2) if so, does JCS do it automatically or there are
> some specific
> settings? (couldn't find anything in the docs)
> 3) Is there a way to 'ask' Purgatory to 'block' any
> subsequent 'put'
> requests until the current 'bounded buffer' (with
> MaxPurgatorySize) is
> dumped to the disk and space in the purgatory
> becomes available again (I
> realized it's quite against the rational why the
> Purgotory was
> introduced in the first place, but read on)
> 
> 
> The thing I need to do is this:
> 
> 1) I need to put, lets say, 100K entries into the
> cache, and keep _ONLY_
> 1K of them readily available in the memory. The rest
> should be
> dynamically load from the disk when required
> (replacing least frequently
> used entries in memory)
> 2) the cache is written into only ONCE, at the
> application startup
> 3) the cache is then accessed (read-only)
> extensively during the
> application life cycle
> 
> 
> The problem I'm having is this:
> 
> 1) with current settings (see below) everything
> works as expected, with
> only one exception:
> 2) the memory consumption with 'unbounded' Purgatory
> size
> (MaxPurgatorySize=-1, correct?) is at least 150M
> higher than a bounded
> one (the values I tried ranged from 100 to 1000),
> even after all data is
> loaded into cache. In my case 100M higher means
> twice as much (300M vs
> 150M)
> 3) while it's unknown right now whether the memory
> would eventually be
> freed-up, I suspect it might not
> 4) with bounded purgatory (MaxPurgatorySize = 1000),
> some entries got
> evicted from the cache before they were written to
> the disk.
> 
> 
> Below are my current settings:
> 
> #----------------------
> jcs.default = DC_SMALL_FOOTPRINT
> jcs.default.cacheattributes =
> org.apache.jcs.engine.CompositeCacheAttributes
> jcs.default.cacheattributes.MaxObjects = 1000
> jcs.default.cacheattributes.MemoryCacheName =
> org.apache.jcs.engine.memory.lru.LRUMemoryCache
> jcs.default.cacheattributes.DiskUsagePatternName =
> UPDATE
> 
>
jcs.auxiliary.DC_SMALL_FOOTPRINT=org.apache.jcs.auxiliary.disk.indexed.I
> ndexedDiskCacheFactory
>
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes=org.apache.jcs.auxiliary.dis
> k.indexed.IndexedDiskCacheAttributes
>
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes.maxKeySize=-1
>
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes.MaxPurgatorySize=-1
>
jcs.auxiliary.DC_SMALL_FOOTPRINT.attributes.DiskPath=jcs
> #----------------------
> 
> Thanks!
> 
> -----Original Message-----
> From: Aaron Smuts [mailto:asmuts@yahoo.com] 
> Sent: Monday, August 28, 2006 9:33 AM
> To: JCS Users List
> Subject: RE: configuration question
> 
> 
> 
> --- Vladimir Olenin <VOlenin@cihi.ca> wrote:
> 
> > Thanks. It seems to work now. I wonder if 'UPDATE'
> > policy is set, does
> > 'MaxPurgatorySize' property has any effect then?
> 
> Yes.  In JCS mythology, purgatory is simply a queue
> on the way to disk,
> where the items can be accessed. 
> It's a map on top of a queue.  All items that go to
> disk, go to the
> queue.  You can set a limit on the queue size
> regardless of the disk
> usage pattern.
> 
> 
> 
> > What's the lifecycle of
> > cache element with UPDATE policy? Does it go just
> between 'memory' and
> 
> > 'dc'? (if not found in memory, load from disk and
> put in memory cache,
> 
> > expiring some entries from memory on the way if
> necessary)?
> 
> When the item is put into the cache for the first
> time it goes to disk
> as well as memory (if the max memory size is greater
> than 0).  If the
> item is later now in memory and is retrieved from
> disk, it will go back
> into memory, but it should will still be on disk. 
> If the item is
> evicted from memory, because, let's say, it falls
> off the LRU, then it
> will NOT be put to disk
> again.  This prevents thrashing.   . . .  
> 
> (I want to make another pattern called PRIMARY where
> the items just go
> to disk until they are accessed yet again . . .)
> 
> > 
> > Thanks for quick answers, Aaron.
> > 
> > Vlad
> > 
> > -----Original Message-----
> > From: Aaron Smuts [mailto:asmuts@yahoo.com]
> > Sent: Monday, August 28, 2006 9:21 AM
> > To: JCS Users List
> > Subject: RE: configuration question
> > 
> > 
> > 
> > --- Vladimir Olenin <VOlenin@cihi.ca> wrote:
> > Sorry, the docs are wrong.  DiskUsagePattern is
> the property but you 
> > have to set it with a "name".  Use
> "DiskUsagePatternName=UPDATE"
> > 
> > 
> > > Trying to do this as per docs, with line
> > > 
> > > jcs.default.cacheattributes.DiskUsagePattern =
> > UPDATE
> > > 
> > > But getting this in the output log:
> > > 
> > > [WARN] Failed to set property diskUsagePattern
> to
> > value "UPDATE".
> > > Conversion to type [short] failed.
> > > 
> > > 
> > > I think 'UPDATE' is a final short variable
> > somewhere in the code.... 
> > > But there is no referral to it in the guides.
> What
> > should it be?
> > > 
> > > -----Original Message-----
> > > From: Aaron Smuts [mailto:asmuts@yahoo.com]
> > > Sent: Friday, August 25, 2006 8:21 PM
> > > To: JCS Users List
> > > Subject: Re: configuration question
> > > 
> > > Purgatory is a queue on which items on the way
> to
> > disk can still be 
> > > accessed.  Setting a limit on purgatory allows
> you
> > to prevent the 
> > > queue from getting too large.  If you set a
> > purgatory to 1000 and then
> > 
> > > quickly add 100,000 items faster than they can
> be
> > written to disk, 
> > > then the queue will certainly hit 1000.  When
> the
> > max purgatory size 
> > > is reached the oldest are dropped.
> > > 
> > > Purgatory is basically a bounded queue with a
> map
> > on top.  There is a 
> > > bit more description here:
> 
=== message truncated ===


---------------------------------------------------------------------
To unsubscribe, e-mail: jcs-users-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jcs-users-help@jakarta.apache.org


Mime
View raw message