asterixdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From abdullah alamoudi <bamou...@gmail.com>
Subject Re: External function dependency problem
Date Thu, 30 Jun 2016 19:06:43 GMT
I will look into that.

@Xikui,
Do you have a reproducable use case where such a behavior can be observed?

Cheers,
Abdullah.

On Thu, Jun 30, 2016 at 7:53 PM, Raman Grover <ramangrover29@gmail.com>
wrote:

> Hi
>
> Each external library has an separately associated custom class loader
> (URLClassLoader) that loads the contained functions and any dependencies
> (jar files) packaged inside the library. The custom loader fits into the
> natural hierarchy with system (classpath based) classloader as the
> immediate parent and bootstrap classloader as the root. As the delegation
> model works, a class loader attempts to load a class only after parent(s)
> in the lineage have failed.
>
> In your specific case, the system class loader is able to find the jackson
> library from the configured classpath and so the library classloader is not
> involved at all for this jar. The problem arising due to different versions
> of a jar present across the hierarchy of classloaders is similar to one
> faced in web containers such as Tomcat or application servers such as
> Websphere where system loaded classes can override any application specific
> classes due to name conflict.
>
> The way you are defining a library is correct. The fix needs to be in the
> way the classes are loaded by AsterixDB (here acting as an application
> server).
>
> I see two ways:-
>
> a) Skipping the system class loader: AsterixDB uses a URLClassloader to
> load libraries. Setting the  parent of the this classloader as null implies
> the bootrstrap classloader (root) as the immediate parent. The bootstrap
> classloader loads rt.jar (java runtime classes) and i18n.jar and these
> definitely do not have conflicts with your library dependencies (if they
> do, not sure why one would override the classes in rt.jar and so we need to
> fix the library). Note that the system classloader is out of the hierarchy
> here and so any other jars loaded by AsterixDB from its classpath (pom.xml)
> would not be visible.
>
> b) Defining a Custom Class loader: Inside AsterixDB, we provide a custom
> implementation of classloader that attempts to load a class prior to
> delegating to the parent. This way, the library classes and packaged
> dependencies override any system level classes from the class path and even
> the classes contained in rt.jar and i18n.jar.
>
> I am opening the discussion here to suggest further alternatives or provide
> preferences.
>
> I have a preference for (a)  (skipping the system class loader) for two
> reasons:
>
> a) it is simpler
>
> b) the other option allows a custom class loader to override classes in
> rt.jar., which is ok but not how classloaders are supposed to work in
> principle.
>
> Regards,
>
> Raman
>
>
>
>
>
>
>
> On Jun 29, 2016 11:07 PM, "Mike Carey" <dtabass@gmail.com> wrote:
>
> > Any classloader experts have suggestions...?
> > On Jun 29, 2016 10:26 PM, "Xikui Wang" <xikuiw@uci.edu> wrote:
> >
> > > Hi Devs,
> > >
> > > We found a problem when trying to build external functions for the
> > > cloudberry demo.
> > >
> > > When the external function is depend on certain library, the library
> that
> > > comes with the external function will be blocked by same library in
> > > AsterixDB. In our case, our external function 'geoTag' uses jackson
> > v2.7.1,
> > > and we packed all dependencies into one single jar. When running
> 'geoTag'
> > > on Asterix, it will call jackson v2.0.0 in AsterixDB which causes
> > > NullPointerException. We have to manually change pom.xml in AsterixDB
> to
> > > fix that.
> > >
> > > We are wondering is that because we load the external function in a
> wrong
> > > way, or this could be one possible interesting problem which is worth
> > > noticing. Thanks.
> > >
> > > Best,
> > > Xikui
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message