river-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peter Firmstone <j...@zeus.net.au>
Subject Re: OSGi RFC 119 Distributed OSGi - (Was [RE: OSGi and Jini])
Date Thu, 23 Jul 2009 04:11:01 GMT
Hi Gregg,

I hear what your saying. 

Can I share some thoughts?

An Object's Class Type is the fully qualified class name + the originating ClassLoader.

HTTP codebase's are part of the problem, the URLClassLoader is fixed in the object's Type
(class identity), which may change over time.  Michael Warres addressed this problem by creating
a dynamic codebase service where the URL was a cryptographic hash of the jar file (stored
data) identity.

Michael made a presentation on Service based codebases, apparently not 
much code was required to implement it.  We cannot directly copy the 
code (interfaces etc) from the presentation due to copyright, although 
we can produce functionally equivalent code.

http://www.jini.org/files/meetings/eighth/presentations/Warres/Warres.pdf

See also River-316

So the dynamic service based codebase could move around, and be offered redundantly.

In addition, we could update/upgrade/replace the Hierarchical based PreferredClassLoader relationship
with a more flexible DynamicClassLoader based on ClassWorlds to segregate incompatible class
packages while granting compatible classes the ability to communicate.  There is BSD licensed
code that we could build on:

http://classworlds.codehaus.org/  ClassWorlds has some very elegant simple models (easy to
code & use) that may help us.

>From the above website: "The |classworlds| model does away with the hierarchy normally
associated with ClassLoaders.  Instead, there is a pool of ClassRealms <http://classworlds.codehaus.org/apidocs/com/codehaus/classworlds/ClassRealm.html>
which can import arbitrary packages from other ClassRealms. Effectively, |classworlds| turns
the old-style hierarchy into a directed graph."

One might give a library its own ClassLoader in each JVM for instance, 
then we could make that library available to the applications / services 
that depended upon it. A later version of that library would have a 
separate ClassLoader so that "Jar hell" or "Classpath Hell" (standalone 
JVM talk) or its distributed equivalent "Class Type Hell" or 
"ClassLoader Hell" are avoided (unfair: ClassLoaders are a blessing in 
disguise, they add another dimension to namespaces).

The new com.sun.jini.tool.classdepend package has functionality to 
record class dependency relationships in an array, this could be 
modified to also store each classes unique sha-1 or md5 hash, as well as 
serialVersionUID's, if they implement Serialisable or Externalizable, 
this dependency array or a similar string hash (similar to your 
suggestion on namespaces) could be returned on request as part of the 
codebase service.

I purchased a copy of the following paper online (Thank's Jim for the 
tip), I found a freely available copy you can all read.  Its called 
Modular Software Upgrades for Distributed Sytems by Sameer Ajmant, 
Barbara Liskov and Liuba Shrira.  It discusses updating services in 
distributed systems.  It is a course grained versioning system.

http://www.pmg.csail.mit.edu/~ajmani/papers/ecoop06-upgrades.pdf

So I'm currently trying to get my head around a new ClassLoader 
framework for classes where a developer knows which objects need to be 
preserved over time, while their underlaying class implementations may 
change, this would be a fine grained versioning system, at the Class 
Level.  The new package com.sun.jini.tools.classdepend can return a 
dependency tree stored in an array, this could be extended to record 
each classes unique sha-1 or md5 hash, as well as serialVersionUID's, if 
they implement Serialisable or Externalizable, it already stores the 
fully qualified class name.  The sha-1 or md5 hash would also form part 
of Security of downloaded code as this could be checked before loading 
the Class file.

Versioning, Identity and preservation of object state / contents over 
time is much harder and is unsolved. Jim Waldo recommends Classes 
implement interfaces, where only the interfaces are used.  This allows 
objects from different ClassLoaders that implement the same interface to 
interact and be interchanged.   For instance if you have a Class called 
Money.class (version 1) and you reimplement its methods and add 
additional methods in Money.class(version 2), the only way the objects 
can be used in the same array etc is if they share a common Interface or 
ancestor class.  If you inherit Money.class (version 1) and call it 
MoneyTree.class you can override all replaced methods and add additional 
methods and it can be used as a money object, however you can't use the 
new additional methods when in company with the original class Money's 
objects, only those that existed prior.

To overcome this problem, I'm thinking about a ClassLoader Versioning 
Framework for Objects we want to distribute and preserve over time, 
preserving their state and contents while retaining the ability to 
upgrade their class files or bytecodes and also changing their type, 
using Interfaces to enable interoperability between objects with 
different types. Serialization can be used to upgrade objects class file 
bytecodes (marshall, unmarshall into a replacement ClassLoader) with 
required visibility granted by what we can build on using ClassWorlds.  
Any objects linking to the VersionedObjects could be strongly linked via 
a reference object that was updated with the VersionedObject's new hard 
reference location, all other objects could be strongly linked to the 
ReferenceObject and retrieve a weak link from the reference object to 
the VersionedObject.  The ClassLoader Versioning Framework would keep a 
weak reference to each ReferenceObject with a Lease, after the Lease 
expires, the ClassLoader would check if that object still existed 
(weakly referenced, may have been garbage collected) and if so, check if 
its Class file has been updated via an Update Service, which returns the 
hash code for the update Class file.   Alternately this could be 
requested via a ClassLoader method,  each time a weak reference is 
requested through the ReferenceObject. The VersioningClassLoader would 
then, with the hashcode, request a codebaseURL from the Codebase Service 
and create the new version in another ClassLoader, the new 
VersionedObject would be strongly referenced by the existing 
ReferenceObject, leaving the old VersionedObject with no strong 
reference, to be Garbage Collected.

All VersionedObject's must implement Serializable.

A Mutable object would require a Transaction Manager.

Identity is more difficult, for objects who's identity is sufficiently 
determined by the equals() and hashCode() methods this should be 
sufficient.  However objects that require a unique identity can be 
broken down again into two types:

1. Immutable objects where Object.equals() doesn't determine object 
Identity.
2. Mutable objects where Object.equals() doesn't determine object Identity.

I think #1 could be handled by an ObjectUniqueID service that provides 
three unique numbers; The time in milliseconds at the time of request 
and two random numbers.  The object would receive this service at 
instantiation time if required.  The likelihood that two random numbers 
and the time in milliseconds would produce a match for the same fully 
quallified class name would be vary rare indeed.

Well #2 would be more complex, this object would need a Transaction 
Manager, and also require the ObjectUniqueID service at the time of 
instantiation.

With this in mind, I have no idea how to instantiate or construct a 
VersionedObject, I haven't yet figured this part out, perhaps it could 
be implemented by:

Interface VersioningClassLoader {

    public ReferenceObject instantiate(Builder builderObject, String 
fullyQualifiedClassName){};

}


Calling methods on the VersionedObject could be done by:
ReferenceObject ob = versionedClassLoaderInstance.instantiate(builder, 
"my.package.classname");
ACommonInterface foo = ob.getWeakRef(); // The ReferenceObject checks 
the lease is valid first.

or to execute some method:

ob.getWeakRef().someMethod();

When all strong links to the ReferenceObject and the VersionedObject it 
points to, go out of scope, the ReferenceObject and the VersionedObject 
can be garbage collected.

Each ClassLoader would be garbage collected after all VersionedObjects 
it manages go out of scope (no strong references left) objects whose 
leases have expired would be migrated to new ClassLoaders, each 
classloader might have a maximum age determined by maximum Lease time +  
a time window during which it can be given new VersionedObjects.

All objects that the VersionedObjects depend upon, would be considered 
supporting objects and would go out of scope once the VersionedObjects 
do, for example a new class file upgrade might dictate another version 
of an  external library, new objects required by the VersionedObject 
would have to be created during unmarshalling and the new class files 
for the library downloaded via the codebase Service, the codebase 
service would also advertise via the service, the dependency tree it 
contains, so the VersioningClassLoader Framework would determine the 
Libraries suitability and make it available via a LibraryClassLoader if 
it isn't already available.

All objects upon unmarshalling would be checked to see if they currently 
exist in memory based on Lease Validity, identity or equality and if so 
the in memory object used.

What I'm talking about is conceptual and experimental, I'm hoping others 
will be able to provide some thoughts / input, assist and see if we 
can't produce something useful along with the changes you've made and 
lessons learned.  Or alternatively tell me I'm totally nuts ;)

Cheers,

Peter.


Gregg Wonderly wrote:
> Niclas Hedhman wrote:
>> OSGi zealots don't like Jini for many reasons, probably in reality
>> driven by politics more than technology. One key technology that has
>> repeatedly been mentioned to me as a show-stopper for even considering
>> Jini in a more central role, is the differences in "service
>> attributes/properties". Jini's 'exact-match' Entry matching is
>> considered inferior of the LDAP expressions of OSGi. Somehow it feels
>> like a weak argument, and I think River today would consider
>> convergence on this point.
>
> The issue which I think is never fully considered is that lookup is 
> based on the Java type system, including complex types, and not based 
> on string matching.  If it was just string matching, we'd have RE 
> support now.  But there is no defined RE syntax for "derived from" or 
> "implements X" etc.  The Java type system provides that.
>
> I will concede that the mechanics of matching in reggie are a 
> reimplementation of the type system semantics, because code is not 
> unmarshalled (as a versioning, code corruption, and security measure, 
> at the least).
>
> I'm more than willing to put together a new lookup service that does 
> provide "string only" lookup of Entry.toString() values.  It would be 
> possible to include new data, taken from the Entry values before they 
> are marshalled for transport.
>
> My changes to reggie to support deferred downloading, include the 
> packaging of all class and interface names as part of the 
> MarshalledInstance so that you can ask if an Entry (or the service 
> itself) "is a" without having to unmarshall it.
>
> Mobile code is non-trivial.  Many of us have experience now, and River 
> has some additional tools that were not in place before 
> (PreferredClassLoader for one) which can "help" manage class 
> dependency trees so that unmarshalled object can avoid being corrupted.
>
> My http://griddle.dev.java.net project illustrates that you can have a 
> "Javaspace" like thing with live code, including matching using code.  
> I separated the keys from the data though so that you can use native 
> types for keys and unmarshalling them doesn't "load" any code.
>
> There are lots of things that I suspect the OSGi camp is only now 
> discovering to be necessary "evils".  Some will be things we can work 
> around or do differently.
>
> Designing a completely new lookup, and getting it into the "Jini 
> world" might be a useful task, but we will only be able to do that one 
> time, I'm guessing.
>
> OSGi was aimed at "everyone you want, can participate" in this VM.  
> Jini was aimed at "everyone who wants to, can run code" in this VM.  
> There are different considerations for each.
>
> Participation in OSGi is selected participation because the container 
> is loaded with what you want to run.  Jini is open participation, 
> because the network is full of things that want to run.  It's the 
> subtle, but important difference in who initiates the installation of 
> the software that makes all the difference.
>
> Gregg Wonderly
>


Mime
View raw message