cayenne-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrus Adamchik <and...@objectstyle.org>
Subject Re: Cayenne object storage / memory usage
Date Thu, 20 Jul 2017 06:50:05 GMT
Ok, some data on the new DataObject structure based on the testing with a real application.


1. UPGRADE:

TL;DR: If you are not doing anything fancy, the upgrade is just regenerating Java classes
with a new template. For special cases read on... 

I did an upgrade of a large old monolithic system, which is a bit too cosy with the old CayenneDataObject
structure. It uses every single utility from cayenne-lifecycle, calls generic property API
a lot, and otherwise takes advantage of the underlying Map structure. It was a good system
to test this upgrade. Here are the instructions based on that experience beyond rerunning
cgen:

* Any vars declared as CayenneDataObject need to be replaced with just DataObject. The new
object is still a DataObject, but inherits from the new BaseDataObject.
* Superclass of any custom superclasses of the app persistent objects needs to be changed
from CayenneDataObject to BaseDataObject.
* Check all direct invocations of 'read|writeProperty[Directly]'. If all of them are using
ORM-mapped property names, you are good.  Otherwise you will need to redefined these methods
to fall back to a Map on unknown property. E.g. put [1] in the custom superclass. Going forward
I think we may fold this code in a Cayenne superclass (HybridDataObject? :)) 
* One particularly nasty extension in cayenne-lifecycle was the one handling "UUID relationships"
(ObjectIdRelationshipHandler and friends ... hopefully not many people are using this). For
each such relationship I had to create an ugly hack [2]. But it seems to work.

2. PERFROMANCE

Now the exciting part. For performance testing I picked a monolithic read-only web service
app with dozens (hundreds?) of endpoints. Essentially a huge query cache constantly which
is refreshed non-stop via Cayenne queries. Lots of object churn and GC. An ideal app to test
memory improvements, and the new structures did not disappoint. My benchmark compared the
same app running under Cayenne 4.0.B1 (old) and 4.1 with field-based objects patch (new) on
Java 7 and Jetty. The app was warmed up to account for class loading and cache initialization,
and was then bombarded with HTTP requests for some time. The results:

* Memory use: new is 49% less than old.
* Time spent in GC (per jstat tool): new is 43% less than old.
* Throughput: new is 27% higher (and climbing as the load rises).

Looks impressive! Mind that these numbers are for the entire web app. Though query cache takes
probably 90% of the app memory, so Cayenne optimization is having such a huge overall impact.
The memory use drop helped in more than one way (can run on a smaller server; less GC means
faster average response times and higher throughput). Just think how much money you can save
on AWS costs! :)

So here is my +1 on making field-based DataObject the default in 4.1.

Andrus

-------
[1] 

private Map<String, Object> values;

@Override
public Object readPropertyDirectly(String propName) {
	return values != null ? values.get(propName) : null;
}

@Override
public void writePropertyDirectly(String propName, Object val) {

	// no synchronization .. this is used for special cases and is hopefully single-threaded
	if(values == null) {
		values = new HashMap<>();
	}

	values.put(propName, val);
}

[2] 

private Factory _uuidFactory;

@Override
public void writePropertyDirectly(String propName, Object val) {
    if(UUID_PROPERTY.equals(propName)) {
        if(val instanceof Factory) {
            _uuidFactory = (Factory) val;
            uuid = null;
            return;
        }
        else {
            _uuidFactory = null;
            uuid = (String) val;
        }
    }

    super.writePropertyDirectly(propName, val);
}

@Override
public Object readPropertyDirectly(String propName) {

    if(UUID_PROPERTY.equals(propName)) {
        if(_uuidFactory != null) {
            return _uuidFactory;
        }
    }

    return super.readPropertyDirectly(propName);
}


Mime
View raw message