Mailing-List: contact user-help@cayenne.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cayenne.apache.org
Received-SPF: pass (athena.apache.org: domain of marcin@ish.com.au designates
 59.167.240.32 as permitted sender)
Message-Id: <35722AB3-F3F2-4726-81DC-036460B7648F@ish.com.au>
From: Marcin Skladaniec <marcin@ish.com.au>
To: user@cayenne.apache.org
In-Reply-To: <9218A4B9-A87D-4E9B-8434-A9FC1780F629@objectstyle.org>
Content-Type: multipart/signed; boundary=Apple-Mail-56-927666022; micalg=sha1;
 protocol="application/pkcs7-signature"
Mime-Version: 1.0 (Apple Message framework v919.2)
Subject: Re: caching documentation
Date: Wed, 5 Mar 2008 22:37:28 +1000
References: <7E9C20F6-10AE-4140-97CE-DEE36CAE1F14@ish.com.au>
 <9218A4B9-A87D-4E9B-8434-A9FC1780F629@objectstyle.org>

--Apple-Mail-56-927666022
Content-Type: text/plain;
	charset=US-ASCII;
	format=flowed;
	delsp=yes
Content-Transfer-Encoding: 7bit

Thank you very much for the reply, I need to read through one more  
time and I'll try to make use of it as well as write some  
documentation down for the website.

Marcin

On 05/03/2008, at 9:20 PM, Andrus Adamchik wrote:

>
> On Mar 4, 2008, at 1:26 AM, Marcin Skladaniec wrote:
>
>> Hi
>>
>> The documentation on caching (http://cayenne.apache.org/doc/caching-and-fresh-data.html 
>>  and http://cayenne.apache.org/doc/object-caching.html) isn't very  
>> comprehensive,
>
> Agreed. There are lots of new features related to caching in 3.0,  
> and we do not communicate them well to the users as of yet.
>
>> it does not answer questions like:
>>
>> - what is actually stored in cache pks? datarows ? objectIds ?
>
> There are two types of cache: object cache [1] and query cache.
>
> * Object cache (stored at ObjectContext): Map<ObjectId, Persistent>  
> (it may not be declared as such, but this is what it is).
> * Object cache (stored at DataDomain... so really a snapshot cache):  
> Map<ObjectId, DataRow>
> * Query cache (stored at ObjectContext, aka LOCAL_CACHE):  
> Map<String, List<Persistent|DataRow>
> * Query cache (stored at DataDomain, aka SHARED_CACHE): Map<String,  
> List<DataRow>
>
>
>> - does caching change when paging is on ?
>
> Yes, there are some caveats, and a few things were tweaked recently.  
> LOCAL_CACHE works (both ROP and two tier). There is no SHARED_CACHE  
> support (and I want to make this more formal - throw an  
> IllegalStateException if pagination and SHARED_CACHE are used  
> together). One reason why I want to do that is that it appeared  
> under ROP as if SHARED_CACHE worked, when it fact things worked  
> differently, as a side effect of the special handling of paginated  
> lists on the ROP server (see below).
>
>
>> - does caching require special measures when used with ROP ?  
>> (meaning the propagation of changes between contexts)
>
> Not really, maybe an understanding of how it is implemented.  
> Paginated list is always cached in the *server* local cache,  
> regardless of the query cache settings. I.e. "LOCAL_CACHE +  
> paginated list + ROP" means caching on both server and client;  
> "NO_CACHE + paginated list + ROP" still means caching on the server.  
> This is done in order to avoid transferring unresolved ID's to the  
> client.
>
>
>> - how to properly use SelectQuery.setCacheGroups()?
>
> Cache groups are ignored unless you use advanced implementations of  
> QueryCache on the server (e.g. OSCache). RefreshQuery can also  
> target cache groups (see below). "cache group" is a mechanism to  
> allow backend code to perform smart cache invalidation without  
> knowing anything about the nature of the queries. E.g. you can have  
> two groups "objects_that_change_often" and  
> "objects_that_rarely_change", corresponding to 2 OSCache  
> invalidation rules, "once per minute" vs. "once per day"... Now when  
> you add new queries, you do not need to change configuration, if  
> they fall into one of the existing "groups"...
>
> So the trick with cache groups is to find common data invalidation  
> patterns in your app. Each repeating pattern becomes a group. This  
> is a logical task, with very little code involved.
>
>> what happens when a query has more than one cache group specified?
>
> Invalidation rules for all groups are combined. I rarely used that  
> in practice, but still think this allows some extra flexibility,  
> e.g. if the same query falls in a broad category and also in a very  
> specific one. E.g. "objects_that_rarely_change" and  
> "objects_that_change_when_event_X_occurs".
>
>> - how long the cache entries sit in the memory, is there a way to  
>> invalidate all cache from time to time ?
>
> Query cache (both shared and local): default mechanism is LRU and no  
> expiration. OSCache allows to configure size and advanced expiration  
> rules per cache group.
>
> Snapshot cache: LRU. Size configurable in the Modeler.
>
> Object cache (server): Unlimited size map with weak references.
>
>
>> - how to invalidate cache using RefreshQuery, the http://cayenne.apache.org/doc/refreshquery.html 
>>  is just a list of suggestions on how it might work in the future.
>
> Yeah, this is not documented properly. I need to poke around a bit  
> more to provide accurate information on RefreshQuery behavior. It  
> was an early idea of cache handling, but I stopped using it in my  
> own apps, as OSCache works beautifully, supports clustering, etc.,  
> etc. And rather importantly - it removes cache management logic from  
> the code (i.e. explicit invalidation vs. configuration-based one).
>
>> Me and Ari are willing to document the caching feature, but we  
>> would need some help.
>
> Awesome! I'd imagine the trick here is to separate everything  
> discussed here into "internal-design-not-relevant-to-the-user" part  
> and "cache-user-guide" part to avoid confusing people and exposing  
> too many implementation details that will likely change over time.
>
> Andrus
>
>
> [1] http://cayenne.apache.org/doc/object-caching.html
>


--Apple-Mail-56-927666022
Content-Disposition: attachment;
	filename=smime.p7s
Content-Type: application/pkcs7-signature;
	name=smime.p7s
Content-Transfer-Encoding: base64

MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIGIzCCAtww
ggJFoAMCAQICEH04uRLie7YYyXPPwOrwracwDQYJKoZIhvcNAQEFBQAwYjELMAkGA1UEBhMCWkEx
JTAjBgNVBAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMTI1RoYXd0ZSBQ
ZXJzb25hbCBGcmVlbWFpbCBJc3N1aW5nIENBMB4XDTA3MTAyMDAxNDg0OVoXDTA4MTAxOTAxNDg0
OVowQzEfMB0GA1UEAxMWVGhhd3RlIEZyZWVtYWlsIE1lbWJlcjEgMB4GCSqGSIb3DQEJARYRbWFy
Y2luQGlzaC5jb20uYXUwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDu7dUT0rWcPhY8
Taf7mqzfUJTad8M5+Z535eVrhnvrPRTQxP/+4xqxykUsSiID4WbD9jZrN9qXt52wbbROlUzu4Psj
E0HXbSgnigGwn8ktbCg3LEH36p2YvkFwwrk9qPE8VNR6hZ+6BtBvJ1lpSkNZCYjOMnXoKwt2gFJv
Iespvi9TgPTNLHJ6mHqIXDBymAJMwQY69Yi91B0pcZpTCeJV3xNoOjkf/FwBWrJJmGOVuwnYgMrJ
k26wM5rRfRgUB2lRv0gTvIhWMV7pT7gkSbpVrvQQAty6HytfWyvSl0WT0K3IIZ5J7PZs5tF8YUzP
XSyucmiH6ufyA9vvpBpQSUCRAgMBAAGjLjAsMBwGA1UdEQQVMBOBEW1hcmNpbkBpc2guY29tLmF1
MAwGA1UdEwEB/wQCMAAwDQYJKoZIhvcNAQEFBQADgYEABFL6WBcmfwdhx346uDsEfWnFp8UDgbT/
x/XfGOQV0/wXgk2LK0Qc5iokICdCRv51oAigBMjFEQD0DLldqxWE0y9fDa/J3HCKox+M0oZfxIQg
9wcpwPftnrf3RyWowUJGoKtTxoiAwYHiQJSHiGKRvDvmjk3e/QLmA102yZm/wtcwggM/MIICqKAD
AgECAgENMA0GCSqGSIb3DQEBBQUAMIHRMQswCQYDVQQGEwJaQTEVMBMGA1UECBMMV2VzdGVybiBD
YXBlMRIwEAYDVQQHEwlDYXBlIFRvd24xGjAYBgNVBAoTEVRoYXd0ZSBDb25zdWx0aW5nMSgwJgYD
VQQLEx9DZXJ0aWZpY2F0aW9uIFNlcnZpY2VzIERpdmlzaW9uMSQwIgYDVQQDExtUaGF3dGUgUGVy
c29uYWwgRnJlZW1haWwgQ0ExKzApBgkqhkiG9w0BCQEWHHBlcnNvbmFsLWZyZWVtYWlsQHRoYXd0
ZS5jb20wHhcNMDMwNzE3MDAwMDAwWhcNMTMwNzE2MjM1OTU5WjBiMQswCQYDVQQGEwJaQTElMCMG
A1UEChMcVGhhd3RlIENvbnN1bHRpbmcgKFB0eSkgTHRkLjEsMCoGA1UEAxMjVGhhd3RlIFBlcnNv
bmFsIEZyZWVtYWlsIElzc3VpbmcgQ0EwgZ8wDQYJKoZIhvcNAQEBBQADgY0AMIGJAoGBAMSmPFVz
VftOucqZWh5owHUEcJ3f6f+jHuy9zfVb8hp2vX8MOmHyv1HOAdTlUAow1wJjWiyJFXCO3cnwK4Va
qj9xVsuvPAsH5/EfkTYkKhPPK9Xzgnc9A74r/rsYPge/QIACZNenprufZdHFKlSFD0gEf6e20Txh
BEAeZBlyYLf7AgMBAAGjgZQwgZEwEgYDVR0TAQH/BAgwBgEB/wIBADBDBgNVHR8EPDA6MDigNqA0
hjJodHRwOi8vY3JsLnRoYXd0ZS5jb20vVGhhd3RlUGVyc29uYWxGcmVlbWFpbENBLmNybDALBgNV
HQ8EBAMCAQYwKQYDVR0RBCIwIKQeMBwxGjAYBgNVBAMTEVByaXZhdGVMYWJlbDItMTM4MA0GCSqG
SIb3DQEBBQUAA4GBAEiM0VCD6gsuzA2jZqxnD3+vrL7CF6FDlpSdf0whuPg2H6otnzYvwPQcUCCT
cDz9reFhYsPZOhl+hLGZGwDFGguCdJ4lUJRix9sncVcljd2pnDmOjCBPZV+V2vf3h9bGCE6u9uo0
5RAaWzVNd+NWIXiC3CEZNd4ksdMdRv9dX2VPMYIDEDCCAwwCAQEwdjBiMQswCQYDVQQGEwJaQTEl
MCMGA1UEChMcVGhhd3RlIENvbnN1bHRpbmcgKFB0eSkgTHRkLjEsMCoGA1UEAxMjVGhhd3RlIFBl
cnNvbmFsIEZyZWVtYWlsIElzc3VpbmcgQ0ECEH04uRLie7YYyXPPwOrwracwCQYFKw4DAhoFAKCC
AW8wGAYJKoZIhvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMDgwMzA1MTIzNzI5
WjAjBgkqhkiG9w0BCQQxFgQUtScgl4ApBUtp9ox0S0gg9l2D/H0wgYUGCSsGAQQBgjcQBDF4MHYw
YjELMAkGA1UEBhMCWkExJTAjBgNVBAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAq
BgNVBAMTI1RoYXd0ZSBQZXJzb25hbCBGcmVlbWFpbCBJc3N1aW5nIENBAhB9OLkS4nu2GMlzz8Dq
8K2nMIGHBgsqhkiG9w0BCRACCzF4oHYwYjELMAkGA1UEBhMCWkExJTAjBgNVBAoTHFRoYXd0ZSBD
b25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMTI1RoYXd0ZSBQZXJzb25hbCBGcmVlbWFpbCBJ
c3N1aW5nIENBAhB9OLkS4nu2GMlzz8Dq8K2nMA0GCSqGSIb3DQEBAQUABIIBAFStyhXjdZB0ebDr
XchLxVwtp4gqlha2D50v5FtmZqFY8J6BSuGHXv6bl4lAKEfoJ+H5pK1hLprcdr0qscjL3ZPMcKNF
YCWExg7wHothO23ji7ps42hyC6WAF1H3t3qXiJu87iW/Eu0Xybu7s5kuhVU/S9EGujpww8NU4xvq
GB2SFrTb8TTMFnqfpNyRvFnt7gQaooCrG3Q76F1MCLo3TNiREhLpzNjOeeOEW3NJ9HlTIW/escfr
lyVjd76gAYGtG8prM0v3E6OHXf4KWyXV+P7ZAbYA9Io8X9OPfeukXRyEv5FdnGjpRvCoaxEF8sYJ
Me3XLSaZs96JLqKY6T6fSn4AAAAAAAA=

--Apple-Mail-56-927666022--