jakarta-jcs-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Roll, Stuart L" <Stuart.R...@ca.com>
Subject Trying to use IndexedDiskCache for a queue
Date Mon, 18 Oct 2010 18:19:01 GMT
I am new to JCS and I thought that IndexedDiskCache might be a good way
to provide a persistent overflow ability to a queue.

I have a need to send messages from one service to another but the
receiving service sometimes gets busy and can't handle more incoming
messages for a while.  I could just keep them in a memory queue but that
would impose storage limits and would not survive a shutdown.

IndexedDiskCache seems to fit the bill for what I want.  All messages
would go into the cache, but since most entries would be removed almost
immediately by a forwarding thread they would never get written.  If the
message flow backs up, entries kept in memory would be written to disk
automatically. When shutting down, the keys get written as well and the
cached items can be made available on startup.

Unfortunately, when I tried to learn about the details of the cache I
couldn't find any detailed explanation of how it actually works.  What's
more, there have apparently been quite a few changes in design between
1.3 and 1.3.3.5-RC.  I'd like to offer a couple of examples of problems
I'm having in hopes that you can educate me (hopefully you won't tell me
I'm an idiot for trying this with JCS!).

I attempted to do some basic testing of the cache using the 1.3.3.5-RC
build but I ran into some problems.  I am directly instantiating the
cache as follows:

		IndexedDiskCacheAttributes attrs = new
IndexedDiskCacheAttributes();
		attrs.setCacheName(CACHE_NAME);
		attrs.setDiskPath(file.getPath());
		cache = new IndexedDiskCache(attrs);

One of the first oddities I found is that calling update() adds an entry
to the cache but getSize() does not reflect that until later when the
item is written to disk.  Is getSize() only supposed to report on the
number of entries on disk?  Shouldn't there be a way to get the number
of entries in the cache?  Calling doUpdate() directly seems to have the
effect I expect (size updated immediately but writes to disk are still
delayed) but that method was made protected since 1.3 and can't be used
directly anymore.  I must not understand the intentions behind
X/doX/processX methods but then again, I did say I looked for doc. ;)  
 
Another issue is that on restart, the cache reloads the keys from disk
but there is no way for the application to find out what keys were
cached!  I suppose I could hack it out of the keyHash through
introspection but that seems like bad form to me.  Shouldn't there be
some way to iterate over the cache keys?  I know I could wrap the cache
to keep a parallel set of keys and manage them separately but that would
be extremely wasteful.

There also doesn't appear to be any way exposed to allow the keys to be
periodically flushed to disk (apart from disposing and recreating the
cache).  This ability would help greatly with persistence in the face of
an unexpected termination (e.g., power off).

I know I'm trying to use the IndexedDiskCache in a way that is a little
bit unusual but I think I've seen some similar ideas in old mailing list
posts.  It is tantalizing to me because so much of what I want is
already there, but I need to work through some difficulties.

You advice and instruction is welcome!

Stuart

---------------------------------------------------------------------
To unsubscribe, e-mail: jcs-users-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jcs-users-help@jakarta.apache.org


Mime
View raw message