river-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peter Firmstone <j...@zeus.net.au>
Subject Re: Thinking Aloud - fundamental challenges of jvm based distributed computing
Date Thu, 17 Nov 2011 02:48:08 GMT
Dan is right, these issues are Jvm issues, perhaps we should take a wait 
and see approach?

Could a service API annotation be of use to assist preferred list 
generation?  @Service

   1. Developer annotates the Service Interface.
   2. Modify classdep to identify Service API (the dependencies of the
      Service Interface identified by the @Service annotation).
   3. Ensure everything other than Service API and it's dependencies are
      preferred by the proxy, perhaps using classdep to generate the
      preferred list, given the proxy.
   4. The developer can remove items from the preferred list to increase
      sharing.

On the one hand I'd like to discourage the use of ClassDep, and 
encourage better structuring of codebase artifacts, to minimise, if not 
eliminate, duplication of classes in jar files.  Perhaps we could 
provide a tool with a new name, that uses the existing ClassDepend code 
(based on ASM) to produce the preferred class list.
  
I'm impartial to Nic's idea also, use reachable URL's for all code, it 
partially solves the codebase annotation loss issue, it doesn't prevent 
the classes being loaded into the wrong classloader, which may also 
cause version conflicts, but it will enable the object to be 
deserialised elsewhere.

Perhaps if we put more focus on codebase annotations.  URL - Universal 
Resource Locator and that's the problem, it's location is hard coded, 
which doesn't allow for inexpensive replication or relocation.

Proxy codebase annotations need to be identity based.  We can discover 
location if we have an identity.

However this in itself causes some issues, the codebase annotation is 
sent with every class, sending large identity key strings could present 
a performance problem.  Then there's compatibility, how do we know the 
jar file is the correct version.

Mike Warres used MD5 checksum's for the identity of a jar file, however 
this is no longer considered secure and using a checksum also prevents 
changes being made to the jar file after deployment, sometimes there may 
be legitimate reasons to do so.

Then we have the Maven option, not everyone is happy about using maven.  
Although it's worth noting that new IDE's now load maven projects 
without requiring configuration on the developers part.

What if we had a URI (URL Scheme) that consisted of a domain or list of 
domains (where to start looking), name and a version (what to look for) 
and we define a version scheme to indicate compatibility, to allow 
upgrades?  Rather than use an MD hash to determine if something has been 
modified in transit, why not sign jar files and use DownloadPermission 
instead?

If we followed Nic's suggestion and used this type of URI scheme for 
codebase annotations, an annotation replaced with an earlier version, 
due to classloader resolution, can be replaced by the latest compatible 
version of the jar archive when it is re-transferred.  Unfortunately 
there is no way at present to prevent.

We could use discovery to locate Codebase Services, from the domain 
provided and ask that service for the name and version.  The codebase 
service could be a smart proxy for maven, or uses any popular type of 
repository, or even just the latest location of an http based 
repository, so we'd need to define a service interface that's relevant 
to all types.  A CodebaseServiceManager, might manage a list of codebase 
services and cache downloaded artifacts.

Perhaps we could use DNS-SD based discovery to locate the codebase 
service?  This would provide both a local network multicast and internet 
DNS-SRV based record search.

Optionally if the codebase service cannot be discovered in the domain 
provided, the search could be expanded to use other domains as well.  
Domains are typically represented by Group's in Jini, although this 
isn't mandatory.

Gregg did some work recently (from memory), with 
CodebaseAccessClassloader, a replacement interface for RMIClassLoader 
that may also be relevant.

+1 for Greg's new container too.   I've briefly looked at the code and 
will have some time soon to study it more in depth.  Greg, is it 
possible to integrate security and configuration as well?  Is some of 
the stuff from Tim Blackman's Jini in a jar of use?

Thoughts, Ideas, problems or better solutions?

Succinct points:

Niclas Hedhman wrote:
> Well, not necessarily. I used http:// URLs as the classpath for the entire
> application (yes, in its own child to the system classloader). Classpath
> annotations worked well. I could imagine that a River container has a
> webserver built--in and serving its own classes to needing clients.
>   

Greg Trasuk wrote:
> Just between us...
>
> The surrogate container I've been working on in
> http://svn.apache.org/viewvc/river/jtsk/skunk/surrogate/
> is only incidentally a container for surrogates.
<SNIP>
>  
>
> For the record, I'm still opposed to the idea of defining "the"
> Jini/River container; I believe the deployment environment is an
> implementation detail, and nothing should prevent the current diversity
> of containers and deployment approaches.  Having said that, one of our
> persistent challenges has been the "rocket scientist" nature of getting
> Jini/River up and running, and I believe that a container approach that
> makes packaging service applications as simple as packaging web
> applications will help immensely.
>   

Dan Creswell wrote:
> True but, as I'm hinting, the problem is preferred classes aren't
> being set up right and that's where we should be focusing IMHO.
>
>   
<SNIP>
> Many of the problems you list ought to be solved in the JDK itself
> rather than fudged around with complex classloader structures (we all
> know where this goes, see any J2EE container for reference). That
> seems like an unlikely possibility so I'd be taking path of least
> resistance which appears to be improving the way we deal in preferred
> classes and establishing some meaningful defaults for setting up the
> necessary manifest entries etc.
>
> You could for example have the proxy share nothing of the platform
> bringing with it it's own versions on the basis that makes future
> versioning problems minimal. Of course it'll make for big .jars as
> downloads but hey ho.
I tend to agree on the ClassLoader structure issues, although I'm 
interested in using a subprocess jvm as an Executor, which eliminates 
the ClassLoader issue, fixes memory isolation and security visibility 
issues, and also might allow for defensive copies (via serialization) of 
objects returned from method calls on smart proxy's.  But this hasn't 
been demonstrated, we don't know how well it performs (memory 
consumption) and it creates unnecessary complexity if Oracle decides to 
address the jvm's long overdue memory isolation and ClassLoader issues, 
then the exercise would be a complete waste of time.

Some interesting Java memory isolation research:

http://www.cs.purdue.edu/homes/peugster/Ribbons/


Mime
View raw message