jakarta-jcs-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Silvano <m...@ipsoft.it>
Subject Re: Indexed Disk Cache safe copying of files
Date Fri, 19 Jun 2009 14:09:55 GMT
Hi Aaron,

locking (at least, locking for writing) should be acceptable in our 
scenario.
If a put finds the cache locked for writing, it can fail without 
creating much harm to the application.
We probably could satisfy our needs this way, just flushing the keys to 
the disk when we want to perform the backup (and maybe, defragmenting ? 
is this needed for
restoring purpose ?)
Do you see any other issue with this approach ?

Best regards,
  Silvano.


Aaron Smuts wrote:
> You might be better off using a remote cache, perhaps backed by MySQL.  
>
> As for the snapshot, there are a few ways it can be done.  I'd have to have to lock the
disk for long.  The best way might be to take a copy of the items.  Then get each and write
it to a new data file, one at a time.  This would be more intensive than simply locking the
datafile, copying it, and dumping the keys.  But the more intensive process requires little
locking.
>
> Aaron
>
> --- On Thu, 6/18/09, Silvano <mele@ipsoft.it> wrote:
>
>   
>> From: Silvano <mele@ipsoft.it>
>> Subject: Re: Indexed Disk Cache safe copying of files
>> To: "JCS Users List" <jcs-users@jakarta.apache.org>
>> Date: Thursday, June 18, 2009, 12:24 AM
>> Hi Aron, Thanks a lot for the
>> interest shown in supporting the issue.
>>
>> This approach seems to be the most applicable to our case.
>> If you think you can work on it, please let we know and we
>> can cooperate and help with testing or supporting
>> implementation or whatever needed.
>>
>> - createSnapshot would be first thing we need, at first
>> glance having some 'location' argument to control directory
>> where to write the consistent copy
>>
>> - loadFromSnapshot might be a plus - however if a
>> consistent snapshot for a region can be moved at ease using
>> Unix commands, a restore from a snapshot could be done by
>> moving phisically the files to the desired location.
>>
>> - timer ... again, this might be handled at the client, but
>> of course would be a plus to configure as a timer
>> We might need to have control over backing up different
>> regions either individually, or several together e.g. in a
>> short sequence so the backup is of diffferent regions is
>> roughly at the same time). We say 'roughly'
>> since the cache does not vary very often and/ or we could
>> limit writes to the cache at the application level for some
>> time if needed to help consistency of the snapshot.
>>
>> Robustness of the snapshot creation would be the most
>> important requirement.
>> in general, we should consider that the files can be large
>> enough (we have so far 1.5 GB of cached data - reason for
>> caching a lot is that the backend is a CMS - rather slow to
>> generate web pages).
>>
>> Silvano.
>>
>>
>> Aaron Smuts wrote:
>>     
>>> The keys are not written until shutdown.  I could
>>>       
>> create a method that could write them to disk and make a
>> copy of the datafile.  It could be called
>> "createSnapShot"  You could call it
>> programatically.  There are details to think
>> through.  Should we try to load from a snapshot on
>> startup?  How many snapshots should we store? 
>> Could we create these via a timer?  I guess this could
>> all be configurable.
>>     
>>> Aaron
>>>
>>>
>>>
>>> ----- Original Message ----
>>> From: Silvano <mele@ipsoft.it>
>>> To: jcs-users@jakarta.apache.org
>>> Sent: Wednesday, June 17, 2009 4:23:52 AM
>>> Subject: Indexed Disk Cache safe copying of files
>>>
>>> Since we have a large cache that needs significant
>>>       
>> time to be populated, we need to be able to copy the files
>> of the regions configured as Indexed Disk Cache between
>> different installations.
>>     
>>> We believe this can be done safely if we turn off the
>>>       
>> jvm, but we would need to copy safely the files while the
>> jvm is running. However it seems that the key files are
>> written only when you turn off the jvm (at least the size
>> changes when we turn off the jvm, so we suspect that they
>> are not synchronized to disk in normal operations).
>>     
>>> We can make sure at application level that nobody
>>>       
>> writes in the cache during the file copy process, but is
>> this sufficient?
>>     
>>> Can anybody recommend a solution to this issue?
>>>
>>> Best regards,
>>>     Silvano.
>>>
>>>
>>>       
>> ---------------------------------------------------------------------
>>     
>>> To unsubscribe, e-mail: jcs-users-unsubscribe@jakarta.apache.org
>>> For additional commands, e-mail: jcs-users-help@jakarta.apache.org
>>>
>>>
>>>       
>> ---------------------------------------------------------------------
>>     
>>> To unsubscribe, e-mail: jcs-users-unsubscribe@jakarta.apache.org
>>> For additional commands, e-mail: jcs-users-help@jakarta.apache.org
>>>
>>>
>>>
>>>    
>>>       
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: jcs-users-unsubscribe@jakarta.apache.org
>> For additional commands, e-mail: jcs-users-help@jakarta.apache.org
>>
>>
>>     
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: jcs-users-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: jcs-users-help@jakarta.apache.org
>
>
>
>   


---------------------------------------------------------------------
To unsubscribe, e-mail: jcs-users-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jcs-users-help@jakarta.apache.org


Mime
View raw message