incubator-cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kelven Yang <kelven.y...@citrix.com>
Subject Re: [DISCUSS] Fix Bloat: Getting rid of autoscanning and using xml for spring DI
Date Thu, 21 Feb 2013 23:02:23 GMT
Rohit,

I'll checkout out your work about using XML instead of @Component.
However, I'm not sure if the memory leak issue is related with auto-scan
at all. I'm now doing a heap analysis and will update with the result

Kelven



On 2/21/13 7:02 AM, "Rohit Yadav" <bhaisaab@apache.org> wrote:

>Okay Chip, posted a comment on the issue.
>
>Hi Kelven, you want to check my fix. Thanks.
>
>Regards.
>
>On Thu, Feb 21, 2013 at 8:17 PM, Chip Childers
><chip.childers@sungard.com> wrote:
>>
>> Can you coordinate with Kelvin on this?  He's got CLOUDSTACK-1276, and
>> had previously mentioned this exact same thing (using the xml file vs.
>> annotations).
>>
>> On Thu, Feb 21, 2013 at 07:26:42PM +0530, Rohit Yadav wrote:
>>> Just to add, we will have separate xmls for every kind of profile, I
>>> can make every plugin have its own spring xml which is copied and
>>> injected to the web.xml comments, suggestions, flames?
>>>
>>> Possible future experiments:
>>> - Spring xmls artefacts based
>>> - Have a way to fix AOP, don't add pointcuts on all classes, methods
>>> - Make DI framework replaceable in CloudStack, I would go for Guice ;)
>>>
>>> Regards.
>>>
>>> On Thu, Feb 21, 2013 at 7:22 PM, Rohit Yadav <bhaisaab@apache.org>
>>>wrote:
>>> > I'm experimenting with removing all @Components and moving beans into
>>> > an xml that could be read by spring to do DI.
>>> > This decreases the load time and
>>> >
>>> > Errors:
>>> > - Abstract classes annotated as spring components
>>> > - Annotations for project which are not included in either build
>>> > profiles: usage, simulator, oss (default) and nonoss
>>> > - Some project don't even build or are not part of any build
>>> > artefacts, looks like the @Component annotation was done in a hurry
>>>:)
>>> > - Interfaces being annotated as spring components
>>> > - Prasanna, Abhi and I spoke with Spring gurus Vijay V and Vaibhav on
>>> > how to fix this bloat, so in applicationContext.xml we do some aop
>>> > stuff to have transaction stuff we get from @DB annotation and this
>>> > needs fixing as it's applying the following for all classes and
>>> > methods (dude bloat, run! :)
>>> >
>>> >   <aop:config proxy-target-class="true">
>>> >     <aop:aspect id="dbContextBuilder"
>>>ref="transactionContextBuilder">
>>> >         <aop:pointcut id="captureAnyMethod"
>>> >             expression="execution(* *(..))"
>>> >         />
>>> >
>>> > Methodology and results:
>>> > I wrote a python program that would go through the whole source code
>>> > and fix all java files (except test, awsapi) who have @Component and
>>> > create bean rules in an xml file. Next I had to manually fix few
>>> > things (see error above) and I got the mgmt server running, the load
>>> > time had improved a lot (little faster now) after disabling
>>> > autoscanning and just using the xml generated. I see the benefit of
>>> > just using the xml and not having spring as a build time dependency
>>> > (we're using @Component in our classes so :).
>>> >
>>> > This was an experiment mostly because a lot of folks are having
>>>spring
>>> > memory issues, especially Prasanna's devcloud-ci cannot run and check
>>> > our commits as memory kills dom0. So, if no one objects may I go
>>>ahead
>>> > and commit it? For me the jvm took about 650MB real memory on osx.
>>> >
>>> > Right now I made it for all build profiles, nonoss being the
>>>superset,
>>> > my experiment can be checked out from here:
>>> > https://github.com/bhaisaab/cloudstack/tree/spring-annotation-to-xml
>>> >
>>> > Regards.
>>> >
>>> > The python program, in case anyone wants to replicate or is curious:
>>> > (defaulter is just a file containing result of: file . | grep java$ >
>>> > defaulter)
>>> > f = open('defaulter', 'r')
>>> > files = f.read().split('\n')[:-1]
>>> > files = filter(lambda x: not x.startswith('./awsapi') and
>>> > x.find('/test/') == -1, files)
>>> > xmldata = ""
>>> > bean = """  <bean id="%s" class="%s"/>\n"""
>>> > xml = """<!--
>>> >   Licensed to the Apache Software Foundation (ASF) under one
>>> >   or more contributor license agreements. See the NOTICE file
>>> >   distributed with this work for additional information
>>> >   regarding copyright ownership. The ASF licenses this file
>>> >   to you under the Apache License, Version 2.0 (the
>>> >   "License"); you may not use this file except in compliance
>>> >   with the License. You may obtain a copy of the License at
>>> >
>>> >   http://www.apache.org/licenses/LICENSE-2.0
>>> >
>>> >   Unless required by applicable law or agreed to in writing,
>>> >   software distributed under the License is distributed on an
>>> >   "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
>>> >   KIND, either express or implied. See the License for the
>>> >   specific language governing permissions and limitations
>>> >   under the License.
>>> > -->
>>> > <beans xmlns="http://www.springframework.org/schema/beans"
>>> >   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>>> >   xmlns:context="http://www.springframework.org/schema/context"
>>> >   xmlns:tx="http://www.springframework.org/schema/tx"
>>> >   xmlns:aop="http://www.springframework.org/schema/aop"
>>> >   xsi:schemaLocation="http://www.springframework.org/schema/beans
>>> >
>>> > http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
>>> >                       http://www.springframework.org/schema/tx
>>> >              
>>>http://www.springframework.org/schema/tx/spring-tx-3.0.xsd
>>> >                       http://www.springframework.org/schema/aop
>>> >
>>> > http://www.springframework.org/schema/aop/spring-aop-3.0.xsd
>>> >                       http://www.springframework.org/schema/context
>>> >
>>> > 
>>>http://www.springframework.org/schema/context/spring-context-3.0.xsd">
>>> >
>>> > %s
>>> > </beans>
>>> > """
>>> > misses = []
>>> > for file in files:
>>> >     name = file.split('/')[-1].split('.')[0]
>>> >     name = name[0].lower() + name[1:]
>>> >     pkgname = ""
>>> >     idx = 0
>>> >     paths = file.split('/')[1:]
>>> >     paths[-1] = paths[-1].split('.')[0]
>>> >     for str in paths:
>>> >         if str == 'org' or str == 'com':
>>> >             pkgname = '.'.join(paths[idx:])
>>> >         idx += 1
>>> >     f = open(file, 'r')
>>> >     data = f.read()
>>> >     f.close()
>>> >     idx = data.find('@Component')
>>> >     if idx == -1:
>>> >         misses.append(file)
>>> >         continue
>>> >     string = '@Component'
>>> >     if data[len(string)+idx] == '(':
>>> >       endidx = data.find(')', len(string)+idx)
>>> >       name = 
>>>data[len(string)+idx+1:endidx].split('=')[-1].replace('"', '')
>>> >       name = name.replace("'", '')
>>> >       data = data[:idx] + data[endidx+2:]
>>> >     else:
>>> >       data = data.replace(string, '')
>>> >     data = data.replace('import
>>> > org.springframework.stereotype.Component;\n', '')
>>> >     f = open(file, 'w')
>>> >     f.write(data)
>>> >     f.close()
>>> >     print name, pkgname
>>> >     xmldata += bean % (name, pkgname)
>>> >     xmldata += "\n"
>>> >
>>> > xml = xml % xmldata
>>> > f = open('./client/tomcatconf/defaultComponentContext.xml.in', 'w')
>>> > f.write(xml)
>>> > f.close()
>>> >
>>> > f = open('misses', 'w')
>>> > f.write('\n'.join(misses))
>>> > f.close()
>>>


Mime
View raw message