incubator-any23-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michele Mostarda <michele.mosta...@gmail.com>
Subject Re: Plugins loading
Date Fri, 17 Feb 2012 15:55:51 GMT
On 17 February 2012 10:21, Simone Tripodi <simonetripodi@apache.org> wrote:

> Hi all guys,
>
> The ML at ASF is the best place where we can discuss ;) Also because
> the more people can access to the info, then easier would be involving
> new volunteers that wants to help :)
>
> Anyway, speaking more about technical details: I had a look at the
> current codebase and even if we restrict the package where performing
> the plugins research, this operation has always the same complexity,
> because it needs to visit the whole ClassPath
>

We can scan only the directory meant to contain plugins, the current CLI
configuration
scans the entire classpath and takes about 2 secs, I expect better
performances for a smaller
set of JARs.


>
> Moreover, it consumes memory: it loads ALL classes for any recognized
> item that has to be passed to the filter, that checks if the current
> item is a plugin or not.
>

Agree, this can be avoided implementing a cache on the classpath, the
PluginManager
can check if the cache is valid and then access it.


>
> Just for the record, have a look at
> <
> http://lemnik.wordpress.com/2007/09/14/why-classpath-scanning-is-bad-for-you/
> >
>
> The Jersey approach is a little different, IIRC they did a little
> trick on <
> http://docs.oracle.com/javase/6.0/docs/api/java/lang/ClassLoader.html#getResources(java.lang.String)
> >
> (and is not efficient as well)
>
> What I am proposing is something really simpler, using the
> ServiceLoader <
> http://docs.oracle.com/javase/6/docs/api/java/util/ServiceLoader.html:>
> that would still allow dynamic definition of the plugins loaded at
> runtime, but since plugins are already known at compile time, we can
> drastically reduce the class loading.
>

That's why Phython guys joke Java developers :)


>
> There are tools as well that will help us on generating
> META-INF/services metadata:
> <http://weblogs.java.net/blog/kohsuke/archive/2009/03/my_project_of_t.html
> >
>
> Ah, and just for the record: I prototyped a general purpose lib for
> scanning the classpath, using fluent APIs, @commons, see
> <https://svn.apache.org/repos/asf/commons/sandbox/meiyo/trunk/> -
> commons sandbox is open to all ASF committers, in the case you are
> interested... ;)
>

Will give a look.


>
> All the best,
> -Simo
>

The best.
Bye
Mic


>
> http://people.apache.org/~simonetripodi/
> http://simonetripodi.livejournal.com/
> http://twitter.com/simonetripodi
> http://www.99soft.org/
>
>
>
> On Fri, Feb 17, 2012 at 1:14 AM, Michele Mostarda
> <michele.mostarda@gmail.com> wrote:
> > Hi Simo,
> >
> >  we scan a set of packages that can be specified programmatically, as
> done
> > by Jersey for example to detect providers.
> > There are several optimizations that can be done like building a local
> > cache to avoid a full rescan every time (for example while
> > running CLI tools).  ATM the best choice is to use the more strictest
> > packages as possible.
> >
> > I would like to spend some time optimizing the PluginManager,
> implementing
> > for example a local cache based on, classpath
> > files modifications. I think we could arrive to a general purpose lib
> > for classpath scanning in Java (that ATM is missing AFAIK).
> >
> > Let me know when you're available for a deeper discussion about it.
> >
> > The best.
> >
> > Mic
> >
> > On 16 February 2012 23:24, Lewis John Mcgibbney
> > <lewis.mcgibbney@gmail.com>wrote:
> >
> >> I'm here Simo, but of little help.. I'm nearly burnt out for the day....
> >>
> >> If we can initiate a discussion it is better than nothing at all I
> suppose.
> >>
> >>
> >> On Thu, Feb 16, 2012 at 10:21 PM, Simone Tripodi
> >> <simonetripodi@apache.org>wrote:
> >>
> >> > Hi all,
> >> >
> >> > I started having a look at the current plugins architecture and just
> >> > noticed we scan the classpath: that is the most expensive operation,
> >> > because no optimization can be applied and it is always linear - that
> >> > also means that more jars plugin we add, the more is memory/time
> >> > consumption.
> >> >
> >> > There is an improvement we can apply switching over the ServiceLoader
> >> > pattern, I need anyway more details about the plugins architecture...
> >> > anyone available?
> >> >
> >> > TIA,
> >> > -Simo
> >> >
> >> > http://people.apache.org/~simonetripodi/
> >> > http://simonetripodi.livejournal.com/
> >> > http://twitter.com/simonetripodi
> >> > http://www.99soft.org/
> >> >
> >>
> >>
> >>
> >> --
> >> *Lewis*
> >>
> >
> >
> >
> > --
> > Michele Mostarda
> > Senior Software Engineer
> > skype: michele.mostarda
> > twitter: micmos
> > mail: me@michelemostarda.com
> > site : http://www.michelemostarda.com
>



-- 
Michele Mostarda
Senior Software Engineer
skype: michele.mostarda
twitter: micmos
mail: me@michelemostarda.com
site : http://www.michelemostarda.com

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message