river-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peter <j...@zeus.net.au>
Subject Re: OSGi
Date Sat, 28 Jan 2017 11:02:59 GMT
Ah yes, that's easy, assuming all classes are Serializable :)

Firstly we register a BundleListener with our OSGi framework and keep 
track of installed Bundles.

So Root, is the object graph root, Classes Child1 and Child2 are visible 
to Root.
The first thing we do is push the ClassLoader of Root onto our stack.
When the stream reads in the fields of Root, the first field child1, it 
will read the stream class information, the class Child1 will resolve 
from Bundle BR, since it is visible from there.
Now we obtain Child1's ClassLoader (which provides a BundleReference to 
Bundle C1) and push it onto our stack.
The stream reads in the fields of Child1, the first field Child1Api, is 
an instance of BImpl, it first attempts to use Bundle C1, however 
assuming that BImpl isn't visible from Bundle C1, we use the codebase 
annotation we read in from the stream (read in for each class), to find 
any installed bundles implied by the annotation's URI.
Upon finding BImpl's bundle from the BundleContext, we first ask the 
bundle to load the class, then we check that the class is assignable to 
field myImpl in Child1, before Child1 is instantiated, using 
class.isAssignableFrom.
If this fails and there are other versions of BImpl installed we might 
iterate through them until we have a match.
If there are no BImpl bundles installed, then we use the annotation to 
load a new Bundle.
At this time BImpl's ClassLoader is pushed onto the stack.
However BImpl has no fields to deserialize, so a new instance of BImpl 
is created and before it is used to set the field myImpl in Child1, 
BImpl's ClassLoader is popped off the stack.
Now that all fields of Child1 have been created, Child1 can be created, 
and bundle C1's ClassLoader is popped off the stack.
The same process is repeated for Child2.

Cheers,

Peter.




On 28/01/2017 7:41 PM, "Michał Kłeczek (XPro Sp. z o. o.)" wrote:
> I fail to see how it could possibly work. Could you walk step-by-step 
> serialize/deserialize with the following object graph:
>
> Bundle API:
> interface Api {}
>
> Bundle BR:
> class Root {
>   Child1 child1;
>   Child2 child2;
>
>   Api getApi() {
>     return isEven(getRandom()) ? child1.impl : child2.impl;
>   }
>
> }
>
> Bundle C1
> class Child1 {
>   Child1Api myImpl;
> }
>
> Bundle C2
> class Child2 {
>   Child2Api myImpl;
> }
>
> Bundle Api1:
> interface Child1Api extends Api {}
>
> Bundle Api2:
> interface Child2Api extends Api {}
>
> Bundle BImpl:
> class Impl implements Child1Api, Child2Api {}
>
> Object graph:
> impl = new BImpl()
> root = new Root(new Child1(impl), new Child2(impl));
>
> Serialize and deserialize root.
>
> Thanks,
> Michal
>
> Peter wrote:
>> So here's how we can put it together:
>>
>> Our OIS contains a stack we can use to track ClassLoader's at each 
>> branch in our serialized object graph.
>>
>>   1. First object being read in by the OIS, check the cache see if the
>>      URI annotation is implied.
>>   2. If yes, use this Bundle to deserialize, place this Bundle's
>>      BundleReference [ClassLoader] on the top of the stack.
>>   3. If no, walk the stack, find the first BundleReference, dereference
>>      it's Bundle, obtain the BundleContext and request a new Bundle
>>      with the new URL.
>>   4. For each field in the current object:
>>
>>        * Try to load it's class from the current Bundle .
>>        * If successful push Bundle's ClassLoader onto stack and return
>>          class.
>>        * Catch ClassNotFoundException, it's likely this is a dependency
>>          injected class the parent object's ClassLoader or Bundle
>>          doesn't have type visibility for.
>>        * Check the Bundle cache to see if the URI annotation is implied.
>>        * If yes, then iterate through each, load a class from this
>>          bundle, check that the returned class can be assigned to the
>>          current field's type.  If not, discard and continue until a
>>          class is found that can be assigned to the current field.
>>        * If not, load the new bundle.
>>        * Push the current Bundle's ClassLoader on the top of the stack.
>>
>>   5. Pop the last (Object in the graph) field's ClassLoader off the 
>> stack.
>>
>> When the process completes we have a resolved object graph with 
>> correct class visiblity.
>>
>> Regards,
>>
>> Peter.
>>
>> On 28/01/2017 1:49 PM, Peter wrote:
>>> Having implemented an ObjectInputStream (OIS), I can share the 
>>> following insights:
>>>
>>>    * The object at the head of the graph is externally visible and
>>>      encapsulates all other objects and primitives.
>>>    * The objects and primitives at the tails of the graph are fields.
>>>    * Circular references in object graphs are unsafe with untrusted
>>>      input...
>>>    * A serialized object graph without circular references is a tree.
>>>    * Fields may have been dependency injected.
>>>    * Each branch in the tree has an object at the head of the branch.
>>>    * The OIS knows the types of fields in the local class as well as
>>>      the types of fields in the deserialized class before it has been
>>>      resolved.
>>>
>>> The information required to deserialized an OSGi graph is in the OIS 
>>> implementation.  The OIS implementation is not visible to the 
>>> extending MarshalInputStream implementation.
>>>
>>> MarshalInputStream contains a ClassLoader field, defaultLoader.
>>>
>>> So the codebase annotation is critical for the identity of the first 
>>> object,
>>>
>>> Regards,
>>>
>>> Peter.
>>>
>>> On 28/01/2017 3:39 AM, Bharath Kumar wrote:
>>>> Yes Peter. Usage of thread context class loader is discouraged in OSGi
>>>> environment.
>>>>
>>>> http://njbartlett.name/2012/10/23/dreaded-thread-context-classloader.html

>>>>
>>>>
>>>> Some of the problems are hard to solve in OSGi environment. For 
>>>> example,
>>>> creating dynamic java proxy from 2 or more interfaces that are 
>>>> located in
>>>> different bundles.
>>>>
>>>> http://blog.osgi.org/2008/08/classy-solutions-to-tricky-proxies.html?m=1

>>>>
>>>>
>>>> This problem can be solved using composite class loader. But it is
>>>> difficult to write it correctly. Because OSGi environment is dynamic.
>>>>
>>>> I believe that it is possible to provide enough abstraction in 
>>>> river code,
>>>> so that service developers don't even require to use context class 
>>>> loader
>>>> in their services.
>>>>
>>>>
>>>>
>>>> Thanks&  Regards,
>>>> Bharath
>>>>
>>>>
>>>> On 27-Jan-2017 6:25 PM, "Peter"<jini@zeus.net.au>  wrote:
>>>>
>>>>> Thanks Gregg,
>>>>>
>>>>> Thoughts inline below.
>>>>>
>>>>> Cheers,
>>>>>
>>>>> Peter.
>>>>>
>>>>>
>>>>> On 27/01/2017 12:35 AM, Gregg Wonderly wrote:
>>>>>
>>>>>> Is there any thought here about how a single client might use 
>>>>>> both an
>>>>>> OSGi deployed service and a conventionally deployed service?
>>>>>>
>>>>> Not yet, I'm currently considering how to support OSGi by 
>>>>> implementing an
>>>>> RMIClassLoaderSPI, similar to how Dennis has for Maven in Rio.
>>>>>
>>>>> I think once a good understanding of OSGi has developed, we can 
>>>>> consider
>>>>> how an implementation could support that, possibly by exploiting 
>>>>> something
>>>>> like Pax URL built into PreferredClassProvider.
>>>>>
>>>>>
>>>>>    The ContextClassLoader is a good abstraction mechanism for 
>>>>> finding “the”
>>>>>> approriate class loader.  It allows applications to deploy a 
>>>>>> composite
>>>>>> class loader in some form that would be able to resolve classes 
>>>>>> from many
>>>>>> sources and even provide things like preferred classes.
>>>>>>
>>>>> Yes, it works well for conventional frameworks and is utilised by
>>>>> PreferredClassProvider, but it's use in OSGi is discouraged, I'd 
>>>>> like to
>>>>> consider how it's use can be avoided in an OSGi env.
>>>>>
>>>>>
>>>>>> In a Java desktop application, would a transition from a background
>>>>>> thread, interacting with a service to get an object from a 
>>>>>> service which is
>>>>>> not completely resolved to applicable loaders still resolve 
>>>>>> correctly in an
>>>>>> EventDispatch Thread?  That event dispatch thread can have the 
>>>>>> context
>>>>>> class loader set on it by the thread which got the object, to be

>>>>>> the class
>>>>>> loader of the service object, to make sure that the resolution of

>>>>>> classes
>>>>>> happens with the correct class loader such that there will not be

>>>>>> a problem
>>>>>> with the service object having one set of definitions and another

>>>>>> service
>>>>>> or the application classpath having a conflicting class 
>>>>>> definition by the
>>>>>> same name.
>>>>>>
>>>>>> I’ve had to spend quite a bit of time to make sure that these 
>>>>>> scenarios
>>>>>> work correctly in my Swing applications.
>>>>>>
>>>>> Have you got more information?  I'm guessing this relates to delayed
>>>>> unmarshalling into the EventDispatch thread.
>>>>>
>>>>> It's early days yet, I'm still working it out what information is 
>>>>> required
>>>>> to resolve the correct ClassLoaders&  bundles, but this is an 
>>>>> important
>>>>> question, Bharath mentioned Entry's can be utilised for versioning 
>>>>> and this
>>>>> seems like a good idea.
>>>>>
>>>>> What follows are thoughts and observations.
>>>>>
>>>>> A bundle can be created from a URL, provided the codebase the URL 
>>>>> refers
>>>>> to has an OSGi bundle manifest, so this could allow any of the 
>>>>> existing URL
>>>>> formats to deliver a proxy codebase for an OSGi framework.  When 
>>>>> OSGi loads
>>>>> the bundle, the package dependencies will be wired up by the local 
>>>>> env.  If
>>>>> the URL doesn't reference a bundle, then we could use Bharath's 
>>>>> approach
>>>>> and subclass the client's ClassLoader, this does make all the clients
>>>>> classes visible to the proxy however, but that would happen anyway 
>>>>> in a
>>>>> standard Jini / River environment.
>>>>>
>>>>> OSGi gives preference to already loaded bundles when resolving 
>>>>> package
>>>>> dependencies, so the client should be careful not to unmarshall 
>>>>> any proxy's
>>>>> that might require a higher version than the bundle already 
>>>>> installed when
>>>>> the client bundle resolved its dependencies.
>>>>>
>>>>> One of the proxy bundle dependencies will be the service api 
>>>>> bundle.  The
>>>>> proxy bundle can limit the service api package / packages to a 
>>>>> specific
>>>>> version or version range, which it could advertise in an Entry.  
>>>>> Boolean
>>>>> logic comparison of Entry's would have to be performed locally, after
>>>>> matching on the service type (this requires delayed unmarshalling to
>>>>> prevent the resolution of unwanted versions).
>>>>>
>>>>> So, the OSGi bundles installed in remote JVM's communicating with 
>>>>> each
>>>>> other, may not be exactly the same version, but should be compatible.
>>>>> Multiple versions of a package or bundle may be available in a 
>>>>> JVM, so we
>>>>> need to ensure we've selected one imported by the proxy.
>>>>>
>>>>> We could define a simple rule, that the first URL in an 
>>>>> annotation, must
>>>>> be the proxy bundle, with all dependencies to be provisioned by 
>>>>> the OSGi
>>>>> Framework, which would allow the same codebase annotation to be 
>>>>> utilised in
>>>>> a non OSGi environment, but this will probably mean the loss of any
>>>>> trailing URL annotations if the proxy's remarshalled.
>>>>>
>>>>> Once the proxy bundle has been loaded, its ClassLoader needs to be 
>>>>> the
>>>>> default for the remaining stream classes (smart proxy fields), 
>>>>> since it
>>>>> knows the most about what dependencies are required, these will be 
>>>>> visible
>>>>> to the proxy's ClassLoader.
>>>>>
>>>>> If a class doesn't have an existing ClassLoader referenced by 
>>>>> annotatation
>>>>> and cannot be resolved from the proxy's Bundle, it probably 
>>>>> requres a new
>>>>> bundle to be resolved.
>>>>>
>>>>> What's next?  Determine what annotations to retrieve from OSGi 
>>>>> Bundles.
>>>>>
>>>>> How PreferredClassProvider currently obtains codebase annotations 
>>>>> (Trivia:
>>>>> the application/system ClassLoader is a URLClassLoader up to Java 
>>>>> 8, but is
>>>>> no longer an instance of URLClassLoader in Java 9):
>>>>>
>>>>> Summary, If ClassLoader:
>>>>>
>>>>>    1. Is local loader? return java.rmi.server.codebase property or 
>>>>> null
>>>>>       if not defined.
>>>>>    2. Is instance of ClassAnnoation, get annotation and return.
>>>>>    3. Is instance of URLClassLoader, get URL's as string and return.
>>>>>    4. else return java.rmi.server.codebase property or null if not 
>>>>> defined.
>>>>>
>>>>>
>>>>> /**
>>>>>       * Returns the annotation string for the specified class loader
>>>>>       * (possibly null).  If check is true and the annotation 
>>>>> would be
>>>>>       * determined from an invocation of URLClassLoader.getURLs() on
>>>>>       * the loader, only return the true annotation if the current
>>>>>       * security context has permission to connect to all of the 
>>>>> URLs.
>>>>>       **/
>>>>>      private String getLoaderAnnotation(ClassLoader loader, 
>>>>> boolean check)
>>>>> {
>>>>>
>>>>>      if (isLocalLoader(loader)) {
>>>>>          return getClassAnnotation(loader);
>>>>>      }
>>>>>
>>>>>          /*
>>>>>       * Get the codebase URL path for the class loader, if it 
>>>>> supports
>>>>>       * such a notion (i.e., if it is a URLClassLoader or subclass).
>>>>>       */
>>>>>      String annotation = null;
>>>>>      if (loader instanceof ClassAnnotation) {
>>>>>          /*
>>>>>           * If the class loader is one of our RMI class loaders, 
>>>>> we have
>>>>>           * already computed the class annotation string, and no
>>>>>           * permissions are required to know the URLs.
>>>>>           */
>>>>>          annotation = ((ClassAnnotation) 
>>>>> loader).getClassAnnotation();
>>>>>
>>>>>      } else if (loader instanceof java.net.URLClassLoader) {
>>>>>          try {
>>>>>          URL[] urls = ((java.net.URLClassLoader) loader).getURLs();
>>>>>          if (urls != null) {
>>>>>              if (check) {
>>>>>              SecurityManager sm = System.getSecurityManager();
>>>>>              if (sm != null) {
>>>>>                  Permissions perms = new Permissions();
>>>>>                  for (int i = 0; i<  urls.length; i++) {
>>>>>                  Permission p =
>>>>>                      urls[i].openConnection().getPermission();
>>>>>                  if (p != null) {
>>>>>                      if (!perms.implies(p)) {
>>>>>                      sm.checkPermission(p);
>>>>>                      perms.add(p);
>>>>>                      }
>>>>>                  }
>>>>>                  }
>>>>>              }
>>>>>              }
>>>>>              annotation = PreferredClassLoader.urlsToPath(urls);
>>>>>          }
>>>>>          } catch (SecurityException e) {
>>>>>          /*
>>>>>           * If access was denied to the knowledge of the class
>>>>>           * loader's URLs, fall back to the default behavior.
>>>>>           */
>>>>>          } catch (IOException e) {
>>>>>          /*
>>>>>           * This shouldn't happen, although it is declared to be
>>>>>           * thrown by openConnection() and getPermission().  If it
>>>>>           * does happen, forget about this class loader's URLs and
>>>>>           * fall back to the default behavior.
>>>>>           */
>>>>>          }
>>>>>      }
>>>>>
>>>>>      if (annotation != null) {
>>>>>          return annotation;
>>>>>      } else {
>>>>>          return getClassAnnotation(loader);
>>>>>      }
>>>>>      }
>>>>>
>>>>> /**
>>>>>       * Returns the annotation string for the specified class loader.
>>>>>       *
>>>>>       *<p>This method is invoked in order to determine the annotation
>>>>>       * string for the system class loader, an ancestor of the system
>>>>>       * class loader, any class loader that is not an instance of
>>>>>       * {@link ClassAnnotation} or {@link URLClassLoader}, or (for an
>>>>>       * invocation of {@link #getClassAnnotation(Class)
>>>>>       * getClassAnnotation(Class)}) a<code>URLClassLoader</code>
 for
>>>>>       * which the current security context does not have the
>>>>>       * permissions necessary to connect to all of its URLs.
>>>>>       *
>>>>>       *<p><code>PreferredClassProvider</code>  implements
this method
>>>>>       * as follows:
>>>>>       *
>>>>>       *<p>This method returns the value of the system property
>>>>>       *<code>"java.rmi.server.codebase"</code>  (or possibly
an 
>>>>> earlier
>>>>>       * cached value).
>>>>>       *
>>>>>       * @param loader the class loader to obtain the annotation 
>>>>> string
>>>>>       * for
>>>>>       *
>>>>>       * @return the annotation string for the class loader, or
>>>>>       *<code>null</code>
>>>>>       **/
>>>>>      protected String getClassAnnotation(ClassLoader loader) {
>>>>>      checkInitialized();
>>>>>      return codebaseProperty;
>>>>>      }
>>>>>
>>>>>
>>>>> Gregg
>>>>>> On Jan 20, 2017, at 6:08 PM, Peter<jini@zeus.net.au>   wrote:
>>>>>>> Looking at your modifications to ServiceDiscoveryManager, I noticed
>>>>>>> you've made changes to set the context ClassLoader prior to 
>>>>>>> calling the
>>>>>>> lookup service.
>>>>>>>
>>>>>>> This is eventually utilised by PreferredClassProvider to lookup
the
>>>>>>> necessary loader to utilise for deserialization of the lookup

>>>>>>> results.
>>>>>>>
>>>>>>> I think if we develop a RMIClassLoader provider for OSGi, we
can 
>>>>>>> avoid
>>>>>>> utilising the context ClassLoader.
>>>>>>>
>>>>>>> Since all OSGi ClassLoader's are instances of BundleReference,

>>>>>>> it's easy
>>>>>>> to utilise OSGi bundle url anotations (I think this needs to

>>>>>>> incorporate
>>>>>>> bundle versions).  I'd also like to utilise Java 9 jrt style
URL's.
>>>>>>>
>>>>>>> Cheers,
>>>>>>>
>>>>>>> Peter.
>>>>>>>
>>>>>>> On 20/01/2017 11:09 PM, Bharath Kumar wrote:
>>>>>>>
>>>>>>>> Thanks Peter for the review.
>>>>>>>>
>>>>>>>> While creating this POC, I tried to make RIO framework as
set 
>>>>>>>> of OSGI.
>>>>>>>> bundles. Rio project extends LookupDiscoveryManager class
in 
>>>>>>>> one of the
>>>>>>>> class .org.rioproject.impl.client.DiscoveryManagementPool.SharedDi
>>>>>>>> scoveryManager.
>>>>>>>> That's why I removed the final modifier.
>>>>>>>>
>>>>>>>>
>>>>>>>> Regarding groovy files,
>>>>>>>> I have made the org.apache.river as system fragment bundle.
So 
>>>>>>>> we can't
>>>>>>>> import any packages from other bundles. But we can use system

>>>>>>>> bundle's
>>>>>>>> packages,. That's why i removed groovy files. If we use these

>>>>>>>> groovy
>>>>>>>> files,
>>>>>>>> we need to import packages from groovy bundle which is not

>>>>>>>> possible
>>>>>>>> here. I
>>>>>>>> will check JGDMS to see how it is used.
>>>>>>>>
>>>>>>>>
>>>>>>>> Thanks&    Regards,
>>>>>>>> Bharath
>>>>>>>>
>>>>>>>>
>>>>>>>> On Fri, Jan 20, 2017 at 6:09 PM, Peter<jini@zeus.net.au>
   wrote:
>>>>>>>>
>>>>>>>> Hi Bharath,
>>>>>>>>> Re your changes (I've found so far):
>>>>>>>>>
>>>>>>>>> LookupDiscoveryManager is non final, I'm interested why?
>>>>>>>>>
>>>>>>>>> BasicInvocationDispatcher, you've set the context class
loader 
>>>>>>>>> around a
>>>>>>>>> block of code, to use the ClassLoader passed in during

>>>>>>>>> construction.
>>>>>>>>> I'm
>>>>>>>>> currently investigating addong methods where ClassLoader
can 
>>>>>>>>> be passed
>>>>>>>>> in
>>>>>>>>> for OSGi.
>>>>>>>>>
>>>>>>>>> Regarding bundle structure, I've restructured the layout
here 
>>>>>>>>> (so you
>>>>>>>>> don't need to delete Groovy config):
>>>>>>>>>
>>>>>>>>> https://github.com/pfirmstone/JGDMS/tree/Maven_build/modularize/JGDMS

>>>>>>>>>
>>>>>>>>>
>>>>>>>>> The full commit history has been retained, so u can see
all 
>>>>>>>>> changes.
>>>>>>>>>
>>>>>>>>> Cheers,
>>>>>>>>>
>>>>>>>>> Peter.
>>>>>>>>>
>>>>>>>>> Sent from my Samsung device.
>>>>>>>>>
>>>>>>>>>    Include original message
>>>>>>>>> ---- Original message ----
>>>>>>>>> From: Bharath Kumar<bharathkumar.a@gmail.com>
>>>>>>>>> Sent: 20/01/2017 09:42:38 pm
>>>>>>>>> To: dev@river.apache.org
>>>>>>>>> Subject: Re: OSGi
>>>>>>>>>
>>>>>>>>> Hello all,
>>>>>>>>>
>>>>>>>>> I have also added a package in org.apache.river bundle
to 
>>>>>>>>> create the
>>>>>>>>> river
>>>>>>>>> service in osgi environment ( Here RIver
>>>>>>>>> uses NonActivatableServiceDescriptor).
>>>>>>>>>
>>>>>>>>> package name is  org.apache.river.start.ext
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> As river bundle is system fragment, i have to remove
>>>>>>>>> the groovy dependency.
>>>>>>>>> So i removed groovy files.
>>>>>>>>>
>>>>>>>>> net.jini.config.Component.groovy
>>>>>>>>>
>>>>>>>>> net.jini.config.GroovyConfig.groovy
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Thanks&    Regards,
>>>>>>>>>
>>>>>>>>> Bharath
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>
>>>
>>
>


Mime
View raw message