commons-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matt Benson <gudnabr...@gmail.com>
Subject Re: [sandbox] class scanning + karma requests
Date Fri, 22 Jul 2011 17:04:27 GMT
On Fri, Jul 22, 2011 at 11:38 AM, Mark Struberg <struberg@yahoo.de> wrote:
> Hoi Simo!
>
> Yup, looks pretty fine from a users perspective. I'm just not sure if this can be done
in a performance intense scenario. We would need to do a few performance tests upfront.
> Remember we often have 50.000++ classes to scan through. And every Class has ~30 methods
and fields w Annotations + method params + Annotations, ...
> If you would really invoke this filter method for each and every structure element, then
you would get performance problems pretty soon.
>
> Of course we need some kind of filtering, but we must be careful and do lots of performance
tests upfront to not propagate an non-performing pattern.

Maybe once the basic scan is complete, the client SPIs should be
called back using a concurrent Executor, where most of the real
filtering can take place.

Matt

>
> LieGrue,
> strub
>
>
> --- On Fri, 7/22/11, Simone Tripodi <simonetripodi@apache.org> wrote:
>
>> From: Simone Tripodi <simonetripodi@apache.org>
>> Subject: Re: [sandbox] class scanning + karma requests
>> To: "Commons Developers List" <dev@commons.apache.org>
>> Date: Friday, July 22, 2011, 3:54 PM
>> Hi Mark!!!
>> For meiyo APIs I'd like to keep I mean end-user APIs; as
>> Matt
>> suggested you, since you didn't hear about the already
>> existing
>> sandbox component before, the most interesting parts of
>> meiyo are the
>> filtering/callback APIs, expressed via EDSL. Have a look at
>> the simple
>> included testcase[1] to see how they make clear its usage
>> to end user
>> - of course they can be improved!!! - and I bet should not
>> a big issue
>> integrating them in your more efficient engine :P
>> Looking forward to hear from you soon, all the best!
>> Simo
>>
>> [1] http://s.apache.org/YIF
>> [2] http://s.apache.org/5Gt
>>
>> http://people.apache.org/~simonetripodi/
>> http://www.99soft.org/
>>
>>
>>
>> On Fri, Jul 22, 2011 at 4:59 PM, Matt Benson <gudnabrsam@gmail.com>
>> wrote:
>> > On Fri, Jul 22, 2011 at 9:51 AM, Mark Struberg <struberg@yahoo.de>
>> wrote:
>> >> Hi Simo!
>> >>
>> >> Yea, taking the best ideas of both projects (meiyo
>> and xbean-finder) would definitlely be a big bonus.
>> >> But to be honest: I think we should rather stay on
>> the xbean-finder API. The reason is that it's already
>> heavily used in Geronimo, OpenEJB, OpenWebBeans and a few
>> other projects here at the ASF and abroad. So we should aim
>> to make the transition as easy as possible. Of course
>> tweaking the API to some degree is surely possible.
>> >> I also have to admit that I haven't heard of meiyo
>> before yesterday, so I don't know it's capabilities.
>> >>
>> >
>> > IMO the most interesting parts of meiyo to take away
>> are the
>> > filtering/callback APIs.  I would think that this is
>> mostly part of
>> > (or at least an option of) the client API e.g. for the
>> single-scan
>> > feature, that we have to rewrite anyway.  So I don't
>> think these goals
>> > are incompatible.
>> >
>> > Matt
>> >
>> >> In any case you are highly welcome to join forces
>> - any idea/opinion/help is welcome!
>> >>
>> >> LieGrue,
>> >> strub
>> >>
>> >> --- On Fri, 7/22/11, Simone Tripodi <simonetripodi@apache.org>
>> wrote:
>> >>
>> >>> From: Simone Tripodi <simonetripodi@apache.org>
>> >>> Subject: Re: [sandbox] class scanning + karma
>> requests
>> >>> To: "Commons Developers List" <dev@commons.apache.org>
>> >>> Date: Friday, July 22, 2011, 2:20 PM
>> >>> Hi all guys!
>> >>> That sounds really interesting!!!
>> >>> I think that bringing the new ideas in
>> existing [meiyo]
>> >>> APIs would
>> >>> allow us releasing a new component soon!
>> >>> Thanks in advance for your help and
>> interesting, looking
>> >>> forward to
>> >>> hear from you soon!!!
>> >>> Simo
>> >>>
>> >>> http://people.apache.org/~simonetripodi/
>> >>> http://www.99soft.org/
>> >>>
>> >>>
>> >>>
>> >>> On Fri, Jul 22, 2011 at 11:41 AM, Mark
>> Struberg <struberg@yahoo.de>
>> >>> wrote:
>> >>> > Hi!
>> >>> >
>> >>> > Jakob Korherr (apacheId jakobk) is also
>> interested and
>> >>> did some work in this area in MyFaces and OWB
>> already.
>> >>> >
>> >>> > There are 2 main parts in this project
>> >>> >
>> >>> > a.) A classpath scanner which does _not_
>> use
>> >>> Class.getDeclaredXxx etc, since this allocates
>> memory in the
>> >>> ClassLoader which cannot get freed up later.
>> In a mid sized
>> >>> project this leads to getting 100MB of
>> PermGenSpace easily!
>> >>> This is what Davids xbean-finder fixes already
>> by just
>> >>> scanning the structure from the bytecode
>> directly.
>> >>> >
>> >>> > b.) Lots of EE apps need to scan the
>> classpath for
>> >>> annotations (e.g. MyFaces, OpenWebBeans,
>> Tomcat, OpenEJB,
>> >>> etc). So this part is done redundantly a few
>> times when
>> >>> starting the server. I hacked a quick API +
>> SPI to do all
>> >>> this in one go. The trick is to introduce a
>> register
>> >>> mechanism where classscan-clients can register
>> their needs
>> >>> before the scan gets performed.
>> >>> >
>> >>> >
>> >>> > We like to do this at commons because
>> it's really
>> >>> decoupled from all the single projects and
>> could be useful
>> >>> for a lot more projects which added some kind
>> of annotation
>> >>> processing lately.
>> >>> >
>> >>> > LieGrue,
>> >>> > strub
>> >>> >
>> >>> > --- On Fri, 7/22/11, Matt Benson <gudnabrsam@gmail.com>
>> >>> wrote:
>> >>> >
>> >>> >> From: Matt Benson <gudnabrsam@gmail.com>
>> >>> >> Subject: [sandbox] class scanning +
>> karma
>> >>> requests
>> >>> >> To: dev@commons.apache.org
>> >>> >> Cc: dblevins@apache.org,
>> >>> "Gerhard Petracek" <gerhard.petracek@gmail.com>
>> >>> >> Date: Friday, July 22, 2011, 4:30 AM
>> >>> >> First, Simo added [meiyo] to the
>> >>> >> sandbox.  Now, some of the guys
>> from
>> >>> >> the various JEE-related communities
>> have expressed
>> >>> interest
>> >>> >> in working
>> >>> >> with class scanning here at
>> Commons.  What we
>> >>> would
>> >>> >> propose to do is
>> >>> >> start with the code of Geronimo's
>> xbean-finder,
>> >>> which scans
>> >>> >> classes by
>> >>> >> reading bytecode to populate
>> meta-structures,
>> >>> thereby
>> >>> >> sparing permgen
>> >>> >> space whenever possible.  We plan to
>> carefully
>> >>> prune
>> >>> >> and polish that
>> >>> >> code, pull in the best ideas from
>> [meiyo] (Simo
>> >>> and/or any
>> >>> >> of the rest
>> >>> >> of you Commons developers are of
>> course welcome
>> >>> to
>> >>> >> participate as
>> >>> >> always!), and add a single-scan SPI
>> strategy to
>> >>> allow the
>> >>> >> majority of
>> >>> >> consumers to share the impact of
>> scanning.  For
>> >>> my own
>> >>> >> part, I hope to
>> >>> >> be able to contribute such that the
>> final API can
>> >>> >> accommodate most if
>> >>> >> not all conceivable use-cases for
>> classpath
>> >>> scanning, while
>> >>> >> not
>> >>> >> becoming too unwieldy.  We'd like to
>> get started
>> >>> ASAP
>> >>> >> (and obviously
>> >>> >> nothing stops me from going ahead and
>> taking the
>> >>> first
>> >>> >> steps).  I
>> >>> >> would also request our new chair
>> grant sandbox
>> >>> karma to:
>> >>> >>
>> >>> >> struberg (Mark Struberg)
>> >>> >> dblevins (David Blevins)
>> >>> >> gpetracek (Gerhard Petracek}
>> >>> >>
>> >>> >> Thanks,
>> >>> >> Matt
>> >>> >>
>> >>> >>
>> >>>
>> ---------------------------------------------------------------------
>> >>> >> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> >>> >> For additional commands, e-mail: dev-help@commons.apache.org
>> >>> >>
>> >>> >>
>> >>> >
>> >>> >
>> >>>
>> ---------------------------------------------------------------------
>> >>> > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> >>> > For additional commands, e-mail: dev-help@commons.apache.org
>> >>> >
>> >>> >
>> >>>
>> >>>
>> ---------------------------------------------------------------------
>> >>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> >>> For additional commands, e-mail: dev-help@commons.apache.org
>> >>>
>> >>>
>> >>
>> >>
>> ---------------------------------------------------------------------
>> >> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> >> For additional commands, e-mail: dev-help@commons.apache.org
>> >>
>> >>
>> >
>> >
>> ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> > For additional commands, e-mail: dev-help@commons.apache.org
>> >
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> For additional commands, e-mail: dev-help@commons.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org


Mime
View raw message