cayenne-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrus Adamchik <and...@objectstyle.org>
Subject Re: Cayenne object storage / memory usage
Date Thu, 20 Jul 2017 11:13:32 GMT
I did mention that it is coming: https://twitter.com/andrus_a/status/887931800705798144

> On Jul 20, 2017, at 1:59 PM, Michael Gentry <blacknext@gmail.com> wrote:
> 
> Sounds sweet.  Would be a great blog post on our new website design once
> that happens, too.  Then tweet to the blog link...
> 
> I'm jealous.  Still getting 3.1 polished off here...
> 
> 
> On Thu, Jul 20, 2017 at 2:50 AM, Andrus Adamchik <andrus@objectstyle.org>
> wrote:
> 
>> Ok, some data on the new DataObject structure based on the testing with a
>> real application.
>> 
>> 1. UPGRADE:
>> 
>> TL;DR: If you are not doing anything fancy, the upgrade is just
>> regenerating Java classes with a new template. For special cases read on...
>> 
>> I did an upgrade of a large old monolithic system, which is a bit too cosy
>> with the old CayenneDataObject structure. It uses every single utility from
>> cayenne-lifecycle, calls generic property API a lot, and otherwise takes
>> advantage of the underlying Map structure. It was a good system to test
>> this upgrade. Here are the instructions based on that experience beyond
>> rerunning cgen:
>> 
>> * Any vars declared as CayenneDataObject need to be replaced with just
>> DataObject. The new object is still a DataObject, but inherits from the new
>> BaseDataObject.
>> * Superclass of any custom superclasses of the app persistent objects
>> needs to be changed from CayenneDataObject to BaseDataObject.
>> * Check all direct invocations of 'read|writeProperty[Directly]'. If all
>> of them are using ORM-mapped property names, you are good.  Otherwise you
>> will need to redefined these methods to fall back to a Map on unknown
>> property. E.g. put [1] in the custom superclass. Going forward I think we
>> may fold this code in a Cayenne superclass (HybridDataObject? :))
>> * One particularly nasty extension in cayenne-lifecycle was the one
>> handling "UUID relationships" (ObjectIdRelationshipHandler and friends ...
>> hopefully not many people are using this). For each such relationship I had
>> to create an ugly hack [2]. But it seems to work.
>> 
>> 2. PERFROMANCE
>> 
>> Now the exciting part. For performance testing I picked a monolithic
>> read-only web service app with dozens (hundreds?) of endpoints. Essentially
>> a huge query cache constantly which is refreshed non-stop via Cayenne
>> queries. Lots of object churn and GC. An ideal app to test memory
>> improvements, and the new structures did not disappoint. My benchmark
>> compared the same app running under Cayenne 4.0.B1 (old) and 4.1 with
>> field-based objects patch (new) on Java 7 and Jetty. The app was warmed up
>> to account for class loading and cache initialization, and was then
>> bombarded with HTTP requests for some time. The results:
>> 
>> * Memory use: new is 49% less than old.
>> * Time spent in GC (per jstat tool): new is 43% less than old.
>> * Throughput: new is 27% higher (and climbing as the load rises).
>> 
>> Looks impressive! Mind that these numbers are for the entire web app.
>> Though query cache takes probably 90% of the app memory, so Cayenne
>> optimization is having such a huge overall impact. The memory use drop
>> helped in more than one way (can run on a smaller server; less GC means
>> faster average response times and higher throughput). Just think how much
>> money you can save on AWS costs! :)
>> 
>> So here is my +1 on making field-based DataObject the default in 4.1.
>> 
>> Andrus
>> 
>> -------
>> [1]
>> 
>> private Map<String, Object> values;
>> 
>> @Override
>> public Object readPropertyDirectly(String propName) {
>>        return values != null ? values.get(propName) : null;
>> }
>> 
>> @Override
>> public void writePropertyDirectly(String propName, Object val) {
>> 
>>        // no synchronization .. this is used for special cases and is
>> hopefully single-threaded
>>        if(values == null) {
>>                values = new HashMap<>();
>>        }
>> 
>>        values.put(propName, val);
>> }
>> 
>> [2]
>> 
>> private Factory _uuidFactory;
>> 
>> @Override
>> public void writePropertyDirectly(String propName, Object val) {
>>    if(UUID_PROPERTY.equals(propName)) {
>>        if(val instanceof Factory) {
>>            _uuidFactory = (Factory) val;
>>            uuid = null;
>>            return;
>>        }
>>        else {
>>            _uuidFactory = null;
>>            uuid = (String) val;
>>        }
>>    }
>> 
>>    super.writePropertyDirectly(propName, val);
>> }
>> 
>> @Override
>> public Object readPropertyDirectly(String propName) {
>> 
>>    if(UUID_PROPERTY.equals(propName)) {
>>        if(_uuidFactory != null) {
>>            return _uuidFactory;
>>        }
>>    }
>> 
>>    return super.readPropertyDirectly(propName);
>> }
>> 
>> 


Mime
View raw message