Mailing-List: contact dev-help@river.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@river.apache.org
MIME-Version: 1.0
In-Reply-To: <CA+JOpGTGbqQYsNCBuPq_TyhXtON9wSKzciGsYjYedYZjUHi6Yg@mail.gmail.com>
References: <0d1b562cdae98262ebbb3bb025b16e95@org.tizen.email> <CA+JOpGTGbqQYsNCBuPq_TyhXtON9wSKzciGsYjYedYZjUHi6Yg@mail.gmail.com>
From: Bharath Kumar <bharathkumar.a@gmail.com>
Date: Wed, 1 Feb 2017 11:13:44 +0530
Message-ID: <CA+JOpGSkpephYjt-skKifvuA_OZV2um4fp+7=rYhhURJvu8aLw@mail.gmail.com>
Subject: Re: OSGi
To: dev@river.apache.org
Content-Type: multipart/alternative; boundary=94eb2c186b463564a805477188ad
archived-at: Wed, 01 Feb 2017 05:43:57 -0000

--94eb2c186b463564a805477188ad
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Peter,
Please note that Package admin service has been replaced by the Bundle
wiring api. So we may need to use this new api.

Thanks & Regards,
Bharath

On 01-Feb-2017 8:50 AM, "Peter" <jini@zeus.net.au> wrote:

Thanks Nic,

So it follows that the caller has visibility to all imported packages from
other Bundle's ClassLoaders.  But not packages from those bundles that
haven't been imported.

Which is the reason I propose switching context with each object in a
serialized graph, to gain the visibility of the current object's bundle
ClassLoader during deserialization, in order to be able to deserialize
implementation classes.

We need the codebase annotation when a class isn't reachable from the
current context.  Each class in a serialization stream is only resolved
once.

The codebase annotation is exact, it has an identity defined by it's
normalized URI, which may be a specific version in a specific location, it
should not contain a version range, or it will cause annotation loss and
prevent loading of new proxy versions as Gregg mentioned earlier.

A proxy class will not be visible from the clients  calling stack. Hence
the proxy's exact bundle will be loaded.  It is then of critical importance
that the service api bundle is resovled, visible to both the client and
proxy.

The way the proxy bundle is wired depends on it's manifest package imports
and the service api version currently loaded in the client jvm.  Hence the
service needs to publish the api package ranges it's proxy imports in an
Entry (we need a new Entry in the Jini platform to support OSGi).

The client must be using the latest currently loaded service api packages
within the proxy's import ranges, as they will be selected by the framework
for the proxy.  There is a special case with the uses package declaration
we need to understand and document, as this may cause selection of an
earlier api package version.

Hence Package Admin Service will need to be utilised to update the client
before unmarshalling the proxy under rare circumstances when more than one
service api version is in use at the client.

The codebase annotation should only be needed during deserialization for
proxy and dependency injected objects.

Proxy's and dependency injected objects will never be resolved to alternate
bundle versions or a range. This also prevents codebase annotation loss.

I think we're getting closer to understanding a possible solution.

Cheers,

Peter.

Sent from my Samsung device.

  Include original message
---- Original message ----
From: Niclas Hedhman <niclas@hedhman.org>
Sent: 01/02/2017 09:57:28 am
To: dev@river.apache.org
Subject: Re: OSGi

As I think you know, the whole purpose of OSGi is to NOT tie the resolution
to Bundles, but to explicitly Exported (with versions) packages. If people
screw up and don't declare the Import/Export package meta data correctly,
then ordinary OSGi may fail just as spectacularly. The difference being
that Java serialization is slightly more sensitive to class changes than
the code itself, and that was an issue from the start. "Back then" it was
quickly realized that "long-term" storage of serialized state was a bad
idea, but "long-term" is relative to code releases and with highly
distributed systems, this is now a reality in networked applications as
well as storing serialized objects to disk. With that in mind, one should
seriously reconsider the status of Java Serialization in one's application,
realize that every serialized class, incl possible exceptions, is a
published API and needs to be treated as such, just as a REST API, SOAP
interface or HTTP itself. Sorry for the rant...

Back to OSGi; even if you really need to tie a class to a particular
bundle, you can do that with attributes and the 'mandatory' directive.

Cheers
Niclas

On Wed, Feb 1, 2017 at 3:29 AM, Micha=C5=82 K=C5=82eczek <michal.kleczek@xp=
ro.biz>
wrote:

> Unfortunately it is not going to work in the general case as I've shown i=
n
> another message.
>
> The issue is that since you do not have influence on how the bundles of
> ServiceProxy and ImplementationDetailServiceProxy are resolved, you canno=
t
> guarantee that ImplementationDetail interface is going to be resolved fro=
m
> the same bundle for both.
>
> Michal
>
> On Tue, 31 Jan 2017 at 06:38, Bharath Kumar <bharathkumar.a@gmail.com>
> wrote:
>
> > With help of some information from service providers,  we can implement
> > this use case in osgi environment easily.
> >
> > Suppose if we split these classes as separate bundles, we will have 2
> sets
> > of bundles.
> >
> >    1. ImplementationDetail.api - contains ImplementationDetail service
> api
> >    class and its dependencies
> >    2. ImplementationDetail.proxy - smart proxy for ImplementationDetail
> >     service
> >    3. ImplementationDetail.service - ImplementationDetail service
> >    implementation
> >    4. ImplementationDetail.bootstrap - service bootstrap bundle for
> > ImplementationDetail
> >    service.
> >
> > ImplementationDetail.bootstrap bundle creates the service and annotate
> the
> > codebase location.
> >
> >
> >    1. forClient.api - contains ForClient service and its dependencies
> >    2. forClient.proxy - smart proxy(ServiceProxy) for ForClient
> > service. It imports
> >    classes from ImplementationDetail.api bundle and forClient.api
bundle.
> >    3. forClientservice - service implementation(ServiceBackend) and
> >    depends on forClient.proxy and forClient.api bundles
> >    4 forClient.bootstrap - service bootstrap bundle for ForClient
> service
> >
> >
> > Here fooClient.proxy dependens on ImplementationDetail service. So it
> can't
> > be created until ImplementationDetail service is available in the
> network.
> > So forClient.bootstrap bundle registers a ServiceTemplate instance into
> > osgi registry. This will help to find ImplementationDetail service from
> the
> > network.
> >
> > Once ImplementationDetail service is available in the network,
> > forClient.bootstrap
> > bundle creates the ServiceBackend and creates the proxy using
> > ImplementationDetail service's proxy. Here we can pass the proxy
> locations
> > of ImplementationDetail.proxy and forClient.proxy bundles.
> >
> >
> > *How to find codebase location of a service proxy?*
> >
> > Before registering the service, we can create an Entry object with code
> > location. This information can be used by any service which will wrap
> > another service proxy. As we know this information, we can create this
> > Entry object.
> >
> >
> > *Codebase information*
> >
> > If we have the below information in the codebase annotation, then
> > RMIClassLoaderSPI can use these information to install or find the
> correct
> > compatible bundle in the client environment.
> >
> >    - Service API bundle symbolic name, its version and location
> >    - Proxy bundle symbolic name , its version and location
> >    - Third party osgi bundles (libraries), their version and locations
> >    (optional).
> >
> >
> > Even If we annotate the JERI based proxy with service api bundle
> > information, we don't even need to patch the below classes.
> >
> >    1. BasicInvocationDispatcher
> >    2. ServiceDiscoveryManager
> >
> > Because, RMIClassLoaderSPI can find the correct compatible service api
> > bundle in client environment  using codebase annotation.
> >
> > I'll create a POC and will share my experiences.
> >
> >
> > On Tue, Jan 31, 2017 at 10:01 AM, Gregg Wonderly <gergg@cox.net> wrote:
> >
> > > The annotation for the exported services/classes is what is at issue
> > > here.  Here=E2=80=99s the perspectives I=E2=80=99m trying to make sur=
e everyone sees.
> > >
> > > 1) Somehow, exported classes from one JVM need to be resolved in
> another
> > > JVM (at a minimum).  The source of those classes
today, is the codebase
> > > specified by the service.  A directed graph of JVMs exchanging classe=
s
> > > demands that all service like JVMs provide a codebase for client like
> > JVMs
> > > to be able to resolve the classes for objects traveling to the client
> > form
> > > the service.  This is nothing we all don=E2=80=99t already know I bel=
ieve.
> > >
> > > 2) If there is a 3rd party user of a class from
one JVM which is handed
> > > objects resolved by a middle man JVM (as Michal is mentioning here),
> > there
> > > is now a generally required class which all 3 JVMs need to be able to
> > > resolve.  As we know, Jini=E2=80=99s current implementation
and basic design is
> > > that a services codebase has to provide a way for clients to resolve
> the
> > > classes it exports in its service implementation.  In the case Michal
> is
> > > mentioning, the demand would be for the middle man service to have th=
e
> > > classes that it wants the 3rd service to resolve, in some part of its
> > > codebase.  This is why I mentioned Objectspace Voyage earlier.  I
> wanted
> > to
> > > use it as an example of a mechanism which always packages class
> > definitions
> > > into the byte stream that is used for sending objects between VMs.
> > Voyager
> > > would extract the class definitions from the jars, wrap them into the
> > > stream, and the remote JVM would be able to then
resolve the classes by
> > > constructing instances of the class using the byte[]
data for the class
> > > definition.
> > >
> > > Ultimately, no matter what the source of the byte[] data for the clas=
s
> > > definition is, it has to be present, at some point in all VMs using
> that
> > > definition/version of the class.  That=E2=80=99s what I am trying to =
say.  The
> > > issue is simply where would the class resolve from?
 I think that class
> > > definition conveyance, between JVMs is something that we have choices
> on.
> > > But, practically, you can=E2=80=99t change =E2=80=9Cannotations=E2=80=
=9D to
make this work.  If
> > the
> > > middle man above is a =E2=80=9Cproxy=E2=80=9D service which bridges t=
wo different
> > networks,
> > > neither JVM on each network would have routing to
get to the one on the
> > > other side of the proxy JVM.  This is why a mechanism like Objectspac=
e
> > > Voyager would be one way to send class definitions defined on one
> network
> > > to another JVM on another network via this proxy service.
> > >
> > > Of course other mechanisms for class conveyance are possible and in
> fact
> > > already exist.  Maven and even OSGi provide class, version oriented
> > > conveyance from a distribution point, into a particular JVM instance.
> > Once
> > > the class definition exists inside of one of those JVMs then we have
> all
> > > the other details about TCCL and creation of proper versions and
> > resolution
> > > from proper class loaders.
> > >
> > > I don=E2=80=99t think we have to dictate that a particular class conv=
eyance
> > > mechanism is the only one.  But, to solve the problem of how to allow
> > > classes hop between multiple JVMs, we have to  specify how that might
> > work
> > > at the level that service instances are resolved and some kind of
class
> > > loading context is attached to that service.
> > >
> > > The reason I am talking specifically about directed graphs of class
> > > loading is because I am first focused on the fact that there is a lot
> > less
> > > flexibility in trying to resolve through a large
collection of specific
> > > classes rather than an open set of classes resolved through a directe=
d
> > > graph of the code execution path which exposes the places and moments
> of
> > > object use in a much more controlled and natural way to me.
> > >
> > > Gregg
> > >
> > > > On Jan 30, 2017, at 9:14 AM, Micha=C5=82 K=C5=82eczek (XPro Sp. z o=
. o.) <
> > > michal.kleczek@xpro.biz> wrote:
> > > >
> > > > It looks to me like we are talking past each other.
> > > >
> > > > Thread local resolution context is needed - we both agree on this.
> > > > What we do not agree on is that the context should be a single
> > > ClassLoader. It has to be a set of ClassLoaders to support situations
> > when
> > > dependencies are not hierarchical.
> > > >
> > > > The use case is simple - I want to implement "decorator" services
> that
> > > provide smart proxies wrapping (smart) proxies of other services.
> > > > I also want to have Exporters provided as dynamic services which
> would
> > > allow my services to adapt to changing network environment.
> > > >
> > > > And I would like to stress - I am actually quite negative about OSG=
I
> > > being the right environment for this.
> > > >
> > > > Thanks,
> > > > Michal
> > > >
> > > > Gregg Wonderly wrote:
> > > >> Maybe you can help me out here by explaining how it is that
> execution
> > > context and class visibility are both handled by OSGi bundles.  For
> > > example, one of my client applications is a desktop environment.  It
> does
> > > service look up for all services registrations
providing a =E2=80=9CserviceUI=E2=80=9D.
> > It
> > > then integrates all of those services into a desktop
view where the UIs
> > are
> > > running at the same time with each one imbedded in a JDesktopPane or =
a
> > > JTabbedPane or a JFrame or JDialog.  There are callbacks from parts o=
f
> > that
> > > environment into my application which in turn is interacting with the
> > > ServiceUI component.  You have AWT event threads
which are calling out,
> > > into the ServiceUIs and lots of other threads of execution which all,
> > > ultimately, must have different class loading environments so that th=
e
> > > ServiceUI components can know where to load code from.
> > > >>
> > > >> It=E2=80=99s exactly TCCL that allows them to know that based on a=
ll the
> other
> > > class loading standards.  The ClassLoader is exactly
the thing that all
> > of
> > > them have in common if you include OSGi bundles as
well.  The important
> > > detail, is that if the TCCL is not used as new ClassLoaders are
> created,
> > > then there is no context for those new ClassLoaders to reference,
> > > universally.
> > > >>
> > > >> The important details are:
> > > >>
> > > >>      1) The desktop application has to be able to prefer certain
> Entry
> > > classes which define details that are presented to the user.
> > > >>      2) When the user double clicks on a services icon, or right
> > clicks
> > > and selects =E2=80=9COpen in new Frame=E2=80=9D, an async worker thre=
ad needs a TCCL
> > > pointing at the correct parent class loader for the service=E2=80=99s
> > > URLClassLoader to reference so that the preferred classes work.
> > > >>      3) Anytime that the AWT Event thread might be active inside o=
f
> > the
> > > services UI implementation, it also needs to indicate the correct
> parent
> > > class loader if that UI component causes other class loading to occur=
.
> > > >>      4) I am speaking specifically in the context of deferred clas=
s
> > > loading which is controlled outside of the service discovery moment.
> > > >>
> > > >>
> > > >>> On Jan 30, 2017, at 4:04 AM, Micha=C5=82 K=C5=82eczek (XPro Sp. z=
 o. o.) <
> > > michal.kleczek@xpro.biz> <mailto:michal.kleczek@xpro.biz> wrote:
> > > >>>
> > > >>> What I think Jini designers did not realize is that class loading
> can
> > > be treated exactly as any other capability provided by a (possibly
> > remote)
> > > service.
> > > >>> Once you realize that - it is possible to provide a kind of a
> > > "universal container infrastructure" where different class loading
> > > implementations may co-exist in a single JVM.
> > > >>
> > > >> That=E2=80=99s precisely what ClassLoader is for.  TCCL is precise=
ly to
> allow
> > > =E2=80=9Csome class=E2=80=9D to know what context to associate newly =
loaded classes
> with,
> > > so that in such an environment, any code can load classes on behalf o=
f
> > some
> > > other code/context.  It doesn=E2=80=99t matter if it is TCCL or some =
other
> class
> > > management scheme such as OSGi bundles.  We are talking about the sam=
e
> > > detail, just implemented in a different way.
> > > >>
> > > >>> What's more - these class loading implementations may be dynamic
> > > themselves - ie. it is a service that provides the
client with a way to
> > > load its own (proxy) classes.
> > > >>>
> > > >>> In other words: "there not enough Jini in Jini itself=E2=80=9D.
> > > >>
> > > >> I am not sure I understand where the short
coming is at then.  Maybe
> > > you can illustrate with an example where TCCL fails
to allow some piece
> > of
> > > code to load classes on behalf of another piece of code?
> > > >>
> > > >> In my desktop application environment, there is a abstract class
> which
> > > is used by each serviceUI to allow the desktop to know if it provides
> the
> > > ability to open into one of the above mentioned JComponent subclasses=
.
> > > That class is preferred and provided and resolved
using the codebase of
> > the
> > > desktop client.  That class loading environment is
then the place where
> > the
> > > service is finally resolved and classes created so that the proxy can
> be
> > > handed to the serviceUI component which ultimately only partially
> > resolves
> > > from the services codebase.
> > > >>
> > > >> It=E2=80=99s this class compatibility which needs to be lightweigh=
t.
> > > >>
> > > >>> We have _all_ the required pieces in place:
> > > >>> - dynamic code loading and execution (ClassLoaders),
> > > >>> - security model and implementation that allows restricting right=
s
> of
> > > the downloaded code,
> > > >>> - and a serialization/deserialization which allows sending
> arbitrary
> > > data (and yes - code too) over the wire.
> > > >>>
> > > >>> It is just the matter of glueing the pieces together.
> > > >>
> > > >> Correct, but it=E2=80=99s a matter of class compatibility where a =
client
> > > environment has to interact with a service and
the serviceUI components
> > > where TCCL excels and providing the ability to create class loaders
> with
> > > the correct parent context, for Java based code.  OSGi introduces the
> > > opportunity for some extra bells and whistles.
But I don=E2=80=99t see that it
> > can
> > > completely eliminate the nature of TCCL and how it was intended to be
> > used.
> > > >>
> > > >>> Thanks,
> > > >>> Michal
> > > >>>
> > > >>>
> > > >>> Gregg Wonderly wrote:
> > > >>>> <snip>
> > > >>>> I am not an OSGi user.  I am not trying to be an OSGi opponent.
> > What
> > > I am trying to say is that I consider all the commentary in those
> > articles
> > > about TCCL not working to be just inexperience and argument to try an=
d
> > > justify a different position or interpretation of
what the real problem
> > is.
> > > >>>>
> > > >>>> The real problem is that there is not one =E2=80=9C
module=E2=80=9D concept in Java
> > > (another one is almost here in JDK 9/Jigsaw).  No one is working
> together
> > > on this, and OSGi is solving problems in a small part of the world of
> > > software.   It works well for embedded, static systems.  I think OSGi
> > > misses the mark on dynamic systems because of
the piecemeal loading and
> > > resolving of classes.  I am not sure that OSGi developers really
> > understand
> > > everything that Jini can do because of the choices made (and not made=
)
> in
> > > the design.  The people who put Jini together had a
great deal of years
> > of
> > > experience piecing together systems which needed to work well with a
> > faster
> > > degree of variability and adaptation to the environment then what mos=
t
> > > people seem to experience in their classes and work environments whic=
h
> > are
> > > locked down by extremely controlled distribution strategies which end
> up
> > > slowing development in an attempt to control everything that doesn=E2=
=80=99t
> > > actually cause quality to suffer.
> > > >>>>
> > > >>>> Gregg
> > > >>>>
> > > >>>>
> > > >>
> > > >
> > >
> > >
> >
>


--=20
Niclas Hedhman, Software Developer
http://polygene.apache.org <http://zest.apacheorg> - New Energy for Java

--94eb2c186b463564a805477188ad--