Mailing-List: contact any23-dev-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: any23-dev@incubator.apache.org
Received-SPF: pass (nike.apache.org: domain of simone.tripodi@gmail.com
 designates 209.85.216.54 as permitted sender)
MIME-Version: 1.0
Sender: simone.tripodi@gmail.com
In-Reply-To: 
 <CAGrL4zh4-vQRPmfPBQtRhEnEesN6=2SoUbK8PnQ=L7zVSBAa0A@mail.gmail.com>
References: 
 <CAAqLGLPV1BcqX+2Z+3MOPMfQLijcowN-auw3d=HDjGtDMqjXNw@mail.gmail.com>
	<CAGaRif30rnmG6cxJTR_d6umZ2-jsjz90KA12-9=cbe=rpEA4ng@mail.gmail.com>
	<CAGrL4zh4-vQRPmfPBQtRhEnEesN6=2SoUbK8PnQ=L7zVSBAa0A@mail.gmail.com>
Date: Fri, 17 Feb 2012 10:21:47 +0100
Message-ID: 
 <CAAqLGLPq0Tj_vYPYiVzbdWBrk-zd6QGysRizii+pEqpkvQgQQQ@mail.gmail.com>
Subject: Re: Plugins loading
From: Simone Tripodi <simonetripodi@apache.org>
To: any23-dev@incubator.apache.org
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Hi all guys,

The ML at ASF is the best place where we can discuss ;) Also because
the more people can access to the info, then easier would be involving
new volunteers that wants to help :)

Anyway, speaking more about technical details: I had a look at the
current codebase and even if we restrict the package where performing
the plugins research, this operation has always the same complexity,
because it needs to visit the whole ClassPath

Moreover, it consumes memory: it loads ALL classes for any recognized
item that has to be passed to the filter, that checks if the current
item is a plugin or not.

Just for the record, have a look at
<http://lemnik.wordpress.com/2007/09/14/why-classpath-scanning-is-bad-for-y=
ou/>

The Jersey approach is a little different, IIRC they did a little
trick on <http://docs.oracle.com/javase/6.0/docs/api/java/lang/ClassLoader.=
html#getResources(java.lang.String)>
(and is not efficient as well)

What I am proposing is something really simpler, using the
ServiceLoader <http://docs.oracle.com/javase/6/docs/api/java/util/ServiceLo=
ader.html:>
that would still allow dynamic definition of the plugins loaded at
runtime, but since plugins are already known at compile time, we can
drastically reduce the class loading.

There are tools as well that will help us on generating
META-INF/services metadata:
<http://weblogs.java.net/blog/kohsuke/archive/2009/03/my_project_of_t.html>

Ah, and just for the record: I prototyped a general purpose lib for
scanning the classpath, using fluent APIs, @commons, see
<https://svn.apache.org/repos/asf/commons/sandbox/meiyo/trunk/> -
commons sandbox is open to all ASF committers, in the case you are
interested... ;)

All the best,
-Simo

http://people.apache.org/~simonetripodi/
http://simonetripodi.livejournal.com/
http://twitter.com/simonetripodi
http://www.99soft.org/


On Fri, Feb 17, 2012 at 1:14 AM, Michele Mostarda
<michele.mostarda@gmail.com> wrote:
> Hi Simo,
>
> =C2=A0we scan a set of packages that can be specified programmatically, a=
s done
> by Jersey for example to detect providers.
> There are several optimizations that can be done like building a local
> cache to avoid a full rescan every time (for example while
> running CLI tools). =C2=A0ATM the best choice is to use the more strictes=
t
> packages as possible.
>
> I would like to spend some time optimizing the PluginManager, implementin=
g
> for example a local cache based on, classpath
> files modifications. I think we could arrive to a general purpose lib
> for classpath scanning in Java (that ATM is missing AFAIK).
>
> Let me know when you're available for a deeper discussion about it.
>
> The best.
>
> Mic
>
> On 16 February 2012 23:24, Lewis John Mcgibbney
> <lewis.mcgibbney@gmail.com>wrote:
>
>> I'm here Simo, but of little help.. I'm nearly burnt out for the day....
>>
>> If we can initiate a discussion it is better than nothing at all I suppo=
se.
>>
>>
>> On Thu, Feb 16, 2012 at 10:21 PM, Simone Tripodi
>> <simonetripodi@apache.org>wrote:
>>
>> > Hi all,
>> >
>> > I started having a look at the current plugins architecture and just
>> > noticed we scan the classpath: that is the most expensive operation,
>> > because no optimization can be applied and it is always linear - that
>> > also means that more jars plugin we add, the more is memory/time
>> > consumption.
>> >
>> > There is an improvement we can apply switching over the ServiceLoader
>> > pattern, I need anyway more details about the plugins architecture...
>> > anyone available?
>> >
>> > TIA,
>> > -Simo
>> >
>> > http://people.apache.org/~simonetripodi/
>> > http://simonetripodi.livejournal.com/
>> > http://twitter.com/simonetripodi
>> > http://www.99soft.org/
>> >
>>
>>
>>
>> --
>> *Lewis*
>>
>
>
>
> --
> Michele Mostarda
> Senior Software Engineer
> skype: michele.mostarda
> twitter: micmos
> mail: me@michelemostarda.com
> site : http://www.michelemostarda.com