cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rohit Yadav <rohit.ya...@shapeblue.com>
Subject Re: Caching / Ehcache
Date Fri, 02 Nov 2018 05:10:42 GMT
Hi Marc-Aurele,


Thanks for starting a discussion thread. I think the GenericDaoBase uses ehcache to build
an internal cache and the rate-limiting plugin uses ehcache, however I'm not sure to what
extent they are used and the impact of removing them on a production management server. Let
me evaluate this soon and get back on this thread (I'll be away next week on holiday).


Thank you for your contribution, it has been a pleasure working with you in the community.
I wish you good luck in your next endeavor.


- Rohit

<https://cloudstack.apache.org>



________________________________
From: Marc-Aurèle Brothier <m@brothier.org>
Sent: Sunday, October 28, 2018 1:39:53 AM
To: dev@cloudstack.apache.org
Subject: Caching / Ehcache

Hi everyone,

(Again as the email formatting has been removed and was hard to read -
I hope it will be better this time).

While trying to lower the DB load for CloudStack I did some long
testing and here are my outcomes for the current cache mechanism in
CloudStack.

I would be interested to hear from people who try to customize the
ehcache configuration in CS. A PR (
https://github.com/apache/cloudstack/pull/2913) is also open to
desactivate (before deleting) ehcache in CS, read below to understand
why.


# Problems

The code in CS does not seem to fit any caching mechanism especially
due to the homemade DAO code. The main 3 flaws are the following:

## Entities are not expected to be shared

There is quite a lot of code with method calls passing entity IDs value
as long, which does some object fetching. Without caching, this
behavior will create distinct objects each time an entity with the same
ID is fetched. With the cache enabled, the same object will be shared
among those methods. It has been seen that it does generate some side
effects where code still expected unchanged entity attributes after
calling different methods thus generating exception/bugs.

## DAO update operations are using search queries

Some part of the code are updating entities based on a search query,
therefore the whole cache must be invalidated (see GenericDaoBase:
public int update(UpdateBuilder ub, final SearchCriteria<?> sc, Integer
rows);).

## Entities based on views joining multiple tables

There are quite a lot of entities based on SQL views joining multiple
entities in a same object. Enabling caching on those would require a
mechanism to link and cross-remove related objects whenever one of the
sub-entity is changed.


# Final word

Based on the previously discussed points, the best approach IMHO would
be to move out of the custom DAO framework in CS and use a well known
one. It will handle caching well and the joins made by the views in the
code. It's not an easy change, but it will fix along a lot of issues
and add a proven / robust framework to an important part of the code.

The work to change the DAO layer is a huge task, I don't know how / who
will perform it.

What are the proposals for a new DAO framework ?


FYI I will stop working for Exoscale at the end of the month, so I
won't be able to tackle such challenge as I won't be working with CS
anymore. I'll try my best to continue looking at the project to give my
insights and share the experienced I have with CS.


Marc-Aurèle


rohit.yadav@shapeblue.com 
www.shapeblue.com
Amadeus House, Floral Street, London  WC2E 9DPUK
@shapeblue
  
 


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message