jakarta-jcs-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stuart Maclean <stu...@apl.washington.edu>
Subject Re: memory cache not written out to disk on shutdown?
Date Wed, 19 Aug 2009 20:59:21 GMT
Hi Jeffrey, this appears to work.  I se t the disk usage pattern to  
UPDATE, and see the .data file growing on disk while the app runs.  I  
also removed any explicit shutdown calls from my code.  When I exit, I  
see the .key file on disk grow, so an internal shutdown listener is  
saving the keys, as Aaron suggested.

So maybe this is the config I need for my situation.  As long as the  
key and data files are in synch I guess I am good to go.

Thanks again

Stuart

On Aug 19, 2009, at 1:42 PM, Jeffrey Kesselman wrote:

> as someone else mentioned, if your concerned about retaining your  
> cached
> data you really shouldn't be waiting til shut down to try to save  
> it.  Use
> UPDATE and save it as you go, which is what i do in my remotes and  
> it works
> fine.
>
>
> On Wed, Aug 19, 2009 at 4:27 PM, Stuart Maclean
> <stuart@apl.washington.edu>wrote:
>
>> Well, I print out some debug in my VM shutdown listener after the  
>> call to
>> CCF.shutdown and the debug appears, so I assume the shutdown call  
>> is firing.
>>
>> I'll install the 1.3.3.5-RC version.  Do I use the existing pom do  
>> you
>> know, or copy the pom to a new one and tweak?
>>
>> Stuart
>>
>>
>> On Aug 19, 2009, at 1:08 PM, Aaron Smuts wrote:
>>
>> You can download the latest temp build and put it in your local maven
>>> repo.  You aren't forced to only use a remote repo with maven.  No  
>>> one would
>>> use maven otherwise.  You wouldn't be able to use your own  
>>> libraries. . . .
>>>
>>> Running in INFO log level is fine.
>>>
>>> "Disk cache not shutdown properly, shutting down now."  Shouldn't  
>>> show up
>>> if shutdown was called.  That's only called if the vm shutdown  
>>> listener
>>> detects the shutdown.  That won't flush memory, it will merely  
>>> write the key
>>> file to disk.  To get the memory contents to disk, you have to do  
>>> an orderly
>>> shutdown.  Are you sure the listener works?
>>>
>>> Can you past in the logs after shutdown is called.
>>>
>>> You can also configure the region to use the  
>>> DiskUsagePatternName=UPDATE.
>>>
>>> Aaron
>>>
>>>
>>> --- On Wed, 8/19/09, Stuart Maclean <stuart@apl.washington.edu>  
>>> wrote:
>>>
>>> From: Stuart Maclean <stuart@apl.washington.edu>
>>>> Subject: Re: memory cache not written out to disk on shutdown?
>>>> To: "JCS Users List" <jcs-users@jakarta.apache.org>
>>>> Date: Wednesday, August 19, 2009, 12:48 PM
>>>> Hi Aaron,
>>>>
>>>> I just configured a log4j logger, appender for
>>>> org.apache.jcs, set to DEBUG,and I get a ton of output in
>>>> the log.  Nothing looks awry, one WARN though.
>>>>
>>>> 2009-08-19 12:39:54,598 13711 WARN   -
>>>> Region [OpenStreetMaps] Disk cache not shutdown properly,
>>>> shutting down now.
>>>> 2009-08-19 12:39:54,598 13711 INFO   - In
>>>> DISPOSE, [OpenStreetMaps] fromRemote [false]
>>>> Re
>>>>
>>>> This issue is driving me mad, it seems random.  I'll
>>>> clear the cache dir, run the app, exit and inspect the cache
>>>> dir and both /key and .data files will have size 0.
>>>> Then I'll run again and things appear to start to work, and
>>>> the disk cache builds up.  It's as if there is some
>>>> race condition in the shutdown process.  yes, I am
>>>> calling CCF.getInstance().shutdown() in my own VM shutdown
>>>> listener.  Is the jcs shutdown process synchronous, or
>>>> does it spawn a new thread, which might be been abandoned by
>>>> my VM?
>>>>
>>>> I am using jcs 1.3 on jdk 1.6.0_14 on  Mac 10.5.
>>>> I am using maven2 for build, so not sure if I can try the
>>>> latest jcs build?
>>>>
>>>> Any help appreciated.
>>>>
>>>> Stuart
>>>>
>>>>
>>>>
>>>> On Aug 19, 2009, at 12:25 PM, Aaron Smuts wrote:
>>>>
>>>> What version are you using?  (I recommend the
>>>>>
>>>> latest temp build.)
>>>>
>>>>>
>>>>> Try shutting down in an orderly fashion.  Call
>>>>>
>>>> shutdown on the CompositeCacheManager.  In any case,
>>>> the disk cache registers a vm shutdown listener.  It
>>>> should write the data.
>>>>
>>>>>
>>>>> Are there errors in the log?
>>>>>
>>>>> Do you see any .data file?  If not, there may be
>>>>>
>>>> a configuration or a file permission problem.  Check
>>>> the logs.  Set the memory size to 0 and see if the data
>>>> goes to disk during normal usage.
>>>>
>>>>>
>>>>> Aaron
>>>>>
>>>>> --- On Wed, 8/19/09, Stuart Maclean <stuart@apl.washington.edu>
>>>>>
>>>> wrote:
>>>>
>>>>>
>>>>> From: Stuart Maclean <stuart@apl.washington.edu>
>>>>>> Subject: memory cache not written out to disk on
>>>>>>
>>>>> shutdown?
>>>>
>>>>> To: jcs-users@jakartaapache.org
>>>>>>
>>>>>> Date: Wednesday, August 19, 2009, 12:04 PM
>>>>>> Hi all, I am trying to use JCS in a
>>>>>> mapping application.  I have a Swing map
>>>>>>
>>>>> tool, which
>>>>
>>>>> retrieves and displays map tile images from eg
>>>>>> openstreetmap.org.  Logically my cache
>>>>>>
>>>>> objects are
>>>>
>>>>> Images, though I am converting to and from int[]
>>>>>>
>>>>> for
>>>>
>>>>> serialization.  My cache keys are a string
>>>>>>
>>>>> encoding of
>>>>
>>>>> the map tile components z,y,x, e.g z-y-x.
>>>>>>
>>>>>> In my cache.ccf, I am specifying properties just
>>>>>>
>>>>> for
>>>>
>>>>> default regions.  I am right in assuming that
>>>>>>
>>>>> my named
>>>>
>>>>> regions in my code 'inherit' their configuration
>>>>>>
>>>>> from the
>>>>
>>>>> defaults?  Is it true also that if I use two
>>>>>>
>>>>> regions,
>>>>
>>>>> R1 and R2, that EACH region gets its own 10
>>>>>>
>>>>> objects
>>>>
>>>>> (MaxObjects) or is 10 the total number of
>>>>>>
>>>>> in-memory cached
>>>>
>>>>> objects?
>>>>>>
>>>>>> What I want to happen is this.  When cache
>>>>>>
>>>>> empty, go
>>>>
>>>>> to map tile server and get tiles.  Store to
>>>>>>
>>>>> JCS cache.
>>>>
>>>>> Mainatin say 50 in memory for each tile 'layer'
>>>>>>
>>>>> (where a
>>>>
>>>>> layer maps to a tile set from one map
>>>>>>
>>>>> server).  When my
>>>>
>>>>> app requires the 51st image, flush LRU to disk
>>>>>>
>>>>> cache. When
>>>>
>>>>> my app exits, serialize the entire memory cache to
>>>>>>
>>>>> disk,
>>>>
>>>>> effectively using the cache as a persistence
>>>>>>
>>>>> engine.
>>>>
>>>>> When restart my app, get Images from local disk
>>>>>>
>>>>> before going
>>>>
>>>>> to remote map server.
>>>>>>
>>>>>> Alas, when I call shutdown (from a VM shutdown
>>>>>>
>>>>> listener,is
>>>>
>>>>> this the problem???) my .key and .data files on
>>>>>>
>>>>> disk never
>>>>
>>>>> seem to be updated  It's as though the in
>>>>>>
>>>>> memory cache
>>>>
>>>>> is never 'flushed' to disk?  Should it be?
>>>>>>
>>>>>> I seem to be missing something here?  Any
>>>>>>
>>>>> help
>>>>
>>>>> appreciated.
>>>>>>
>>>>>> Stuart
>>>>>>
>>>>>> My cache.ccf is here...
>>>>>>
>>>>>> jcs.default=DC
>>>>>>
>>>>>> jcs 
>>>>>> .default 
>>>>>> .cacheattributes=org.apache.jcs.engine.CompositeCacheAttributes
>>>>
>>>>> jcsdefault.cacheattributes.MaxObjects=10
>>>>>>
>>>>>> jcs 
>>>>>> .default 
>>>>>> .cacheattributes 
>>>>>> .MemoryCacheName=org.apache.jcsengine.memory.lru.LRUMemoryCache
>>>>
>>>>>
>>>>>> jcs.default.elementattributes.IsEternal=true
>>>>>>
>>>>>> jcs.default.elementattributes.MaxLifeSeconds=1000000
>>>>
>>>>> jcs.default.elementattributes.IdleTime=1800
>>>>>> jcs.default.elementattributes.IsSpool=true
>>>>>> jcs.default.elementattributes.IsRemote=true
>>>>>> jcs.default.elementattributes.IsLateral=true
>>>>>>
>>>>>>
>>>>>> jcs 
>>>>>> .auxiliary 
>>>>>> .DC=org.apache.jcs.auxiliary.disk.indexed.IndexedDiskCacheFactory
>>>>
>>>>>
>>>>>> jcs 
>>>>>> .auxiliary 
>>>>>> .DC 
>>>>>> .attributes 
>>>>>> =org.apache.jcs.auxiliary.disk.indexed.IndexedDiskCacheAttributes
>>>>
>>>>>
>>>>>> jcs.auxiliary.DC.attributes.DiskPath=${user.home}/.sgDesktop/ 
>>>>>> map/cache3
>>>>
>>>>> jcs.auxiliary.DC.attributes.MaxKeySize=100000
>>>>>>
>>>>>> jcsauxiliary.DC.attributes.MaxPurgatorySize=100000
>>>>
>>>>>
>>>>>> #jcs.auxiliary.DC..attributes.ShutdownSpoolTimeLimit=60
>>>>
>>>>>
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>
>>>>> To unsubscribe, e-mail: jcs-users-unsubscribe@jakarta.apache.org
>>>>>> For additional commands, e-mail: jcs-users- 
>>>>>> help@jakarta.apache.org
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>
>>>>> To unsubscribe, e-mail: jcs-users-unsubscribe@jakarta.apache.org
>>>>> For additional commands, e-mail: jcs-users-help@jakarta.apache.org
>>>>>
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: jcs-users-unsubscribe@jakarta.apache.org
>>>> For additional commands, e-mail: jcs-users-help@jakarta.apache.org
>>>>
>>>>
>>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: jcs-users-unsubscribe@jakarta.apache.org
>>> For additional commands, e-mail: jcs-users-help@jakarta.apache.org
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: jcs-users-unsubscribe@jakarta.apache.org
>> For additional commands, e-mail: jcs-users-help@jakarta.apache.org
>>
>>
>
>
> -- 
> ~~ Microsoft help desk says: reply hazy, ask again later. ~~


---------------------------------------------------------------------
To unsubscribe, e-mail: jcs-users-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jcs-users-help@jakarta.apache.org


Mime
View raw message