cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kelven Yang <kelven.y...@citrix.com>
Subject RE: Hibernate
Date Fri, 29 Jun 2012 17:06:59 GMT
The discussion is going a bit far away from the original topic, CloudBridge project does have
certain usage on Hibernate, but it is very minimum. Just less than a dozen of Dao classes
needs to be replaced, Rajesh and I had a discussion a while ago, whether to just use direct
JDBC or to use JdbcTemplate offered from Spring.

As of the concerns about eager fetch or lazy loading in ORM framework, I'm sure Hibernate
or other ORM people can give you a number of good practices you should follow, the same applies
to the problem of long application session and short persistent transaction, I know that there
are some good practices out there. We seems to argue here some of the most fundamental design
principals of ORM layer. These kind of debates seems to belong to Hibernate or other ORM forums

David, your mention about MangoDB usage seems to be very interesting, could you elaborate
more details about the problem you were facing and why MangoDB can solve it nicely?

Kelven


> -----Original Message-----
> From: Frank Zhang [mailto:Frank.Zhang@citrix.com]
> Sent: Thursday, June 28, 2012 6:37 PM
> To: cloudstack-dev@incubator.apache.org
> Subject: RE: Hibernate
> 
> >
> > Very interesting.
> >
> > So after the read of this explanation and the architecture and coding
> > implication of the usage of Hibernate my question stay the same ....
> >
> > Does the ORM approach based on a relational DB is the good way ?
> > The usage of a Document Oriented Database seem more in adequacy with
> > the problem to solve.
> >
> > No ?
> 
> I have to say I haven't dug much into Document DB.
> I simply took a look at key-value store and similar no-sql DB.
> I wonder how does this kind of DB do complicated search? Let's the  VM is
> serialized as you said
> 
> > > > *{*
> > > > * name: "myVM",*
> > > > * memory: 1024*
> > > > * network: [{eth0: 195.21.24.12}, {eth1: 12.35.15.28}**]*
> > > > * storage: [{local: [{sdb ....}]}]*
> > > > *}*
> > > > *
> 
> How can I get the vm whose eth0=195.21.24.12? Does search relay on text
> search engine like Lucent?
> 
> >
> 
> >
> >
> > On Thu, Jun 28, 2012 at 8:32 PM, Frank Zhang <Frank.Zhang@citrix.com>
> > wrote:
> >
> > > Hi David:
> > >        I think I can answer part of your question, about graph object.
> > >        Actually CloudStack's database entity the xxxVO you see does
> > > have graph object, however it's not using the typical way which is
> > > well known as composition (e.g VMInstance.NetworkVO.NicVO), we use
> > > id which is the primary key in database to composite object graph.
> > >        Current popular ORM, Hibernate or JPA, use one-to-many,
> > > many-to-one, many-to-many .... or new annotation(JPA2) like
> > > MapKeyJoinColumn to make object graph transparent just as you
> > mentioned.
> > > However, this doesn't fit CloudStack's requirement in terms of huge
> > > number of objects.
> > >               ORM offers to two ways to fetch relational object:
> > > eagerly of lazy. The Eagerly mode definitely won't work for
> > > CloudStack, let me cite an example, assume we have a beautiful graph
> > object hierarchy:
> > >               HostVO
> > >                      |___ VMInstanceVO[]
> > >                                              |_________NicVO[]
> > >
> > >              Now once you get a HostVO, you can transparently browse
> > > vms running on it and even get nic information of VM. However, it
> > > doesn't
> > work.
> > > Let's say you have 10000 hosts, each host run 10 ~ `50 VMs, each VM
> > > has  1 ~ 3 nics. With eager fetching, a listAllHost operation is
> > > going to blow up memory, there are half millions object loaded in
> > > memory and most of them are useless.
> > >
> > >              Lazy mode seemingly resolve this problem, however,
> > > Hibernate and JPA only allow lazy load when your entity is still in
> > > persistence context, according to their saying,  the object must be
> > > attached.  This works pretty well in tradition web application, open
> > > the persistence context when request comes in and close it when
> > > request finishes.  However, in CloudStack, a request like DeployVM
> > > is made up of multiple operations such as preparing storage,
> > > preparing network ... some of these operations (e.g. preparing
> > > storage) are long run, it's impossible to keep a persistence opening
> > > so long. And most of operations are based on theadpool where
> > > persistence context is hard to propagate because it's based on
> threadlocal.
> > > Well, one solution is recommended by hibernate, using detach to
> > > remove entity from persistence and attach it back when needs. IMO,
> > > this is a quite annoying way that brings much trouble to programmer.
> > > Let's say I write a function receiving an entity as parameter:
> > >             Void printHost(HostVO vo) {
> > >                       printVMsonHost(vo.vms)
> > >             }
> > >             Can somebody tell me the HostVO is attached or detached
> now?
> > > no one knows.  You have to call some function of persistence to
> > > judge if the HostVO is attached, if not, attach it back. Otherwise
> > > you will be greeted by a EntityIsNotManaged exception. It ends up
> > > with you have to do boring judge/attach at everywhere you play with
> > > entity object because you don't know if it is attached. Ok, even we
> > > don't mind it, the attach operation is quite annoying. JPA/Hibernate
> > > has a method "detach" entity, but they don't have the opposite method
> to 'attach'
> > > an entity.  Simply google "jpa attach", there are tons of questions
> > > about proper way to re-attach an object back to persistence. The
> > > usual way is to call merge(), but it has the side effect that if you
> > > change the entity accidentally, your change would be written to
> > > database, this is
> > the source of mysterious bug.
> > >
> > >            In CloudStack, you can use id(the primary key of
> > > relational
> > > object) to fetch your composition on demand, though it's not as
> > > intuitive as directly wiring whole relational object in code, it
> > > does its job well in practice.
> > >
> > > > -----Original Message-----
> > > > From: David Avenante [mailto:d.avenante@gmail.com]
> > > > Sent: Thursday, June 28, 2012 4:24 PM
> > > > To: cloudstack-dev@incubator.apache.org
> > > > Subject: Re: Hibernate
> > > >
> > > > Ok I try ;)
> > > >
> > > > As far I can see in the code the majority of the Object have no
> > > relations and
> > > > are simple VO
> > > >
> > > > Many of object use IMHO a bad pattern (DTO, VO ...)
> > > >
> > > > For example :
> > > >
> > > > public class UserVmVO extends VMInstanceVO implements UserVm { ...
> > > >
> > > > There is NO relation between objects. Object are State.
> > > > *The power of Hibernate is to try for the user to make transparent
> > > > the management of a graph object* *There is NO graph object in
> > > > Cloudstack as far I can see.*
> > > > *
> > > > *
> > > > *In the other side I think the Business Domain of the System
> > > > Management
> > > is
> > > > more document oriented.* *For exemple a VM can be represented by a
> > > > simple JSON structure:*
> > > > *
> > > > *
> > > > *{*
> > > > * name: "myVM",*
> > > > * memory: 1024*
> > > > * network: [{eth0: 195.21.24.12}, {eth1: 12.35.15.28}**]*
> > > > * storage: [{local: [{sdb ....}]}]*
> > > > *}*
> > > > *
> > > > *
> > > > *The force of the approach is to have a schema less support. If we
> > > > need
> > > to
> > > > add attribute to a VM no need to migrate de DB.* *In many case
> > > > some part of the document can be exposed in JSON format directly
> > > > to the view (UI).*
> > > > *
> > > > *
> > > > *We have used MongoDB in on of my project (we used hibernate
> > > > before) and the return in term of usage in very very good.* *In
> > > > many many case,
> > > see
> > > > the data as document is a big win.*
> > > > *
> > > > *
> > > > *
> > > > *
> > > >
> > > >
> > > >
> > > >
> > > > On Thu, Jun 28, 2012 at 5:53 PM, Kevin Kluge
> > > > <Kevin.Kluge@citrix.com>
> > > > wrote:
> > > >
> > > > > Rajesh, can you provide some rationale for this choice versus
> > > > > other options.
> > > > >
> > > > > -kevin
> > > > >
> > > > > > -----Original Message-----
> > > > > > From: Rajesh Battala [mailto:rajesh.battala@citrix.com]
> > > > > > Sent: Wednesday, June 27, 2012 10:44 PM
> > > > > > To: cloudstack-dev@incubator.apache.org
> > > > > > Subject: RE: Hibernate
> > > > > >
> > > > > > Hi,
> > > > > >
> > > > > > I had started working on this issue. As Hibernate is LGPL we
> > > > > > cannot use
> > > > > this in
> > > > > > our Apache repo.
> > > > > > I had discussed with Chiradeep and Kelven.
> > > > > >
> > > > > > Am looking at replace Hibernate with Spring Framework
> > > > > > simpleJDBCTemplate.
> > > > > >
> > > > > > The Spring Framework is released under version 2.0 of the
> > > > > > Apache License http://www.springsource.org/spring-framework
> > > > > >
> > > > > > Thanks
> > > > > > Rajesh Battala
> > > > > >
> > > > > >
> > > > > > > -----Original Message-----
> > > > > > > From: Chiradeep Vittal [mailto:Chiradeep.Vittal@citrix.com]
> > > > > > > Sent: Thursday, June 28, 2012 10:45 AM
> > > > > > > To: cloudstack-dev@incubator.apache.org
> > > > > > > Cc: cloudstack-dev@incubator.apache.org
> > > > > > > Subject: Re: Hibernate
> > > > > > >
> > > > > > > The ORM in the AWS module is 90% used by S3.
> > > > > > > The dependency is mostly  abstracted by a DAO layer; there
> > > > > > > is another dependency on transactions. I believe Rajesh
B is
> > > > > > > already working on this aspect and there is a bug open
on it.
> > > > > > >
> > > > > > > --
> > > > > > > Chiradeep
> > > > > > >
> > > > > > > On Jun 27, 2012, at 21:52, "David Nalley" <david@gnsa.us>
> wrote:
> > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > On Jun 28, 2012, at 12:45 AM, Sheng Liang
> > > > > > > > <Sheng.Liang@citrix.com>
> > > > > > > wrote:
> > > > > > > >
> > > > > > > >>> In short, I see three options (please comment
if you see
> > > > > > > >>> more)
> > > 1.
> > > > > > > >>> Rip out
> > > > > > > hibernate and replace with some other ORM 2. Make the AWS
> > > > > > > API bits an optional non-default part of the build.
> > > > > > > >> 3. Declare that hibernate is a system requirement
for
> > > > > > > >> CloudStack
> > > > > > > >>
> > > > > > > >> I prefer option #1. It is the cleanest. I don't
think it
> > > > > > > >> will be very difficult to
> > > > > > > rip out Hibernate.
> > > > > > > >>
> > > > > > > >> Sheng
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > That is my personal inclination as well, though I
am
> > > > > > > > somewhat reticent to
> > > > > > > say so, since I am not doing any of the work to rip and
> replace.
> > > > > > > At the same time choice of ORM is a big issue. I know,
for
> > > > > > > instance that Alex was looking into finding another ORM
for
> > > > > > > the
> > > rest of
> > > > CloudStack.
> > > > > > > When I initially looked at the Hibernate issue, Prachi
told
> > > > > > > me she thought it was about 2 weeks worth of work.
> > > > > > > >
> > > > > > > >
> > > > > > > > --David
> > > > >
> > >

Mime
View raw message