river-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dan Creswell <dan.cresw...@gmail.com>
Subject Re: Stargazing Was: Re: river.jar
Date Wed, 05 Jan 2011 08:48:59 GMT
And...

<snip>


>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>> Okay, so if we're going that way, we all should read Mike Warres'
>>>>>> paper
>>>>>> way
>>>>>> back in the day that covers much of the classloader ball of mud,
>>>>>> preferred
>>>>>> class loading and outstanding issues:
>>>>>>
>>>>>> http://labs.oracle.com/techrep/2006/smli_tr-2006-149.pdf
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>> Damn good suggestion, I was actually going over it again yesterday.
>>>>>
>>>>> Code base annotation loss is a big problem, when non preferred classes
>>>>> are
>>>>> resolved by the application class loader, that and code base location
>>>>> changes over time and codebase configuration problems.
>>>>>
>>>>> Preferred class loading for preferred classes doesn't follow the rule
>>>>> of
>>>>> delegating class loading first to the parent class loader.
>>>>>
>>>>> Preferred class loading is used when proxy classes also exist in the
>>>>> parent
>>>>> class loader.
>>>>>
>>>>> The best option appears to be to limit the classes in the application
>>>>> class
>>>>> loader to jini platform classes, this minimizes the classes that need
>>>>> to
>>>>> be
>>>>> preferred.
>>>>>
>>>>> One problem with using a child class loader for jini platform
>>>>> implementation classes, is some of these classes may invariably be
>>>>> serialized with api classes from the platform and need to be
>>>>> deserialized,
>>>>> if they're not visible to the application class loader...  I've been
>>>>> going
>>>>> over these classes one by one, I'm not finding any serializable classes
>>>>> yet...  I'm starting to think a child class loader for the platform
>>>>> implementation would be an unnecessary complication.
>>>>>
>>>>> Codebase services was intended to be a solution to codebase migration
>>>>> and
>>>>> configuration issues, the codebase service was found using discovery.
>>>>>
>>>>> I've been having some thoughts about a custom URL handler, that
>>>>> utilises
>>>>> DNS-SRV discovery to locate codebases.  Each domain could provide a
>>>>> list
>>>>> of
>>>>> codebase servers, these could be redundant.
>>>>>
>>>>> A custom URL format could have the following information:
>>>>>
>>>>>  * The domain in which to discover the codebase server.
>>>>>  * The jar archive name (or other file type name).
>>>>>  * A message digest of the file.
>>>>>  * A file size limit. (download is aborted after this limit is
>>>>> exceeded)
>>>>>  * A hashcode of the Server's Subject's certificate (the cert itself
>>>>>   could be too large)
>>>>>
>>>>> The Certificate hashcode provides an identity for the URL (which
>>>>> provides
>>>>> a
>>>>> reasonable probability of being unique when combined with other
>>>>> information), we might consider using a Subject to uniquely identify
a
>>>>> ClassLoader namespace, to avoid class sharing between proxies with
>>>>> identical
>>>>> code, but different server subjects.  The Subject certificate hashcode
>>>>> would
>>>>> need to be checked as part of the verification process, albeit after
>>>>> unmarshalling.
>>>>>
>>>>> The problem here is we're bumping into the limitations of the java
>>>>> platform, which the missing Isolates API was designed to address,
>>>>> identical
>>>>> classes could be used by separate identities in separate isolates.
>>>>>
>>>>> If we didn't have a hashcode relating to the server subject, would it
>>>>> be
>>>>> reasonable to expect all servers from the same domain to have the same
>>>>> server Subject?
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>> No real comment other than to say, feels like a narrowing assumption
>>>> that
>>>> could provide a nasty surprise later so I would be inclined not to make
>>>> it.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>> My thoughts exactly, I wanted to see if someone else thought the same
>>> way.
>>>
>>> This is precisely what would occur if two different proxy's with
>>> different
>>> Subject's used the same code base, how could we tell their ClassLoaders
>>> apart?  This might occur if we used maven to provision the codebase or
>>> used
>>> DNS-SRV discovered codebase servers.
>>>
>>>
>>>
>>>
>> Okay so I'm not sure maven and such are the trigger's for the above. It
>> would be some conscious decisions someone had made about packaging and
>> deployment, wouldn't it?
>>
>> If two services and thus two proxies have the same codebase, someone is
>> choosing to either:
>>
>> (1) Have multiple instances of the same service run off the same codebase
>> -
>> not sure of the value except maybe for load-handling. Single codebase for
>> each service is potentially a single point of failure.
>>  (2) Have multiple services of different types sharing a codebase - well
>> this
>> could happen if:
>>
>> (a) We choose to leave all classes sitting under a URL in unjar'd state
>> (we
>> all know that means lots of roundtrips during class resolution etc and so
>> generally it's not done) or
>> (b) We're keeping multiple codebases inside of a single .jar (which
>> increases the size of the download required for class resolution etc and
>> makes version management a difficult affair as touching one service
>>
>>
>
> Or (3) Use DNS-SRV to locate codebases in a domain, such that the URL is
> the domain, not an individual IP address and DNS-SRV is used to locate one
> of a number of redundant codebase servers that can provide the same jar
> file.   In this case, the location is the Domain, since multiple services
> might use the same code to provide the same service to multiple clients, it
> is possible for a client to consume more than one service and it is possible
> that for some reason someone might want to have multiple Subjects for their
> Jini Services from one domain, utilising common code or libraries.  DNS
> spoof attacks can be avoided using message digests.
>

Yep.


>
> So do you think a Certificate would be better than a hashcode?  If so I'm
> happy to go with that.
>
>
At least on first blush, I don't think it makes much of a difference and
would say hashcode would be the default on performance alone. Assuming the
performance gain is actually significant. Given how often we download and
install code in the average client lifecycle I'm not sure it's worth the
worry and would thus say, consider the cert itself.


> If Services have the same Subject, then there is no danger of two distinct
> service object's sharing the same classes and static variables, since
> Permission is granted to Principals, that the Subject has.  It is only when
> run as a Subject after authentication that the proxy gain's Permissions.  It
> could be argued that the unmarshalling period of the second proxy is a
> window where an unauthenticated proxy has a thread, to transfer information
> back to it's host from sensitive static variables, that have been populated
> by the first proxy.  However the first proxy and second proxy have to use
> the same code, the second proxy can't do something unpredictable unless the
> code is not trusted, but in this case it is, since the first proxy trusts
> the code and the client trust the first proxy, so the only thing an
> attacking second proxy can do is, deserialize a large amount of data,
> causing a stack overflow error.
>
> The advantage of DNS-SRV discovery for codebase servers is that codebases
> and services can be deployed separately and have codebase server redundancy.
>
> You're right, I'm trying to solve a problem that doesn't exist, at least
> not yet...
>
> For versioning, a proxy might include the version number in the name of
> it's jar file as Dennis has previously suggested.  We could provide a
> refresh operation on the proxy that remarshalled it's current state and
> updated its codebase annotation, then unmarshalled it in another ClassLoader
> with an updated codebase, all while the client holds a reference to the
> proxy (a reflective proxy), which internally is changed to refer to the new
> proxy.  The original proxy is then garbage collected.
>

Generally, one way or another for versioning to work, the codebase must be
version sensitive. You can do that using various tricks one of which is a
version number in the .jar. I have on many an occasion felt that a
centralised codebase server is nonsense and that each service should have
its own embedded server to handle codebase requests. When that happens,
versioning is kind of implicitly dealt with.


>
> The new codebase might include new functionality that a Service UI might
> take advantage of.
>
> Another possibility is having a reflective verifier proxy in the Marshalled
> Instance, where the client platform is provided as a parameter, and the
> remote end advises a codebase annotation that is more suited to the local
> platform (might take advantage of some local platform feature).
>
> We might use a very simple client that looks for Services with Service UI,
> where the Service UI, is the Service, in this way software could be
> extremely dynamic, Registrar's in Domain's could be discovered using DNS-SRV
> and Unicast Discovery V2, users could browse  services from a variety of
> clients with different UI requirements.  Gregg's Jini desktop, might become
> the cloud desktop.
>
> Hence the subject change to stargazing.
>
> Cheers,
>
> Peter.
>
>  potentially impacts another).
>>
>> Now in case of (1) I cannot see any reason why one would have different
>> subjects for each of those services. In essence they are the same service
>> in
>> all about UUID so housing their proxies in the same classloader on a
>> client
>> doesn't seem like a problem.
>>
>> In case (2), I think the obvious way to bundle a service, one jar for -dl
>> and one jar for back-end is likely the prevailing one and these cases
>> though
>> they could occur would not be recommended practice.
>>
>> For maven, I can see how we could certainly build archetypes and such to
>> support the recommended bundling approach. For DNS SRV, there are means by
>> which codebase paths (.jars and such) can be put into the records such
>> that
>> we could still construct a unique codebase to pull from.
>>
>> In essence I think we might be solving a non-problem...
>>
>> But is a Certificate hashcode enough?
>>
>>
>>> My reasoning for using a Certificate hashcode, rather than a certificate:
>>>
>>>  1. Certificates are large, and would make Codebase URL debugging
>>>    output harder to understand.
>>>
>>>
>>>
>>
>> Unless one builds tools to support this case. If it's a dominating case,
>> that's what we'd have to do in any case I reckon so I'm not fearful.
>>
>>
>>
>>
>>>  2. A Certificate cannot be checked until after proxy verification, it
>>>    would be unlikely that two different certificates would share the
>>>    same hashcode and codebase and be trusted by the client, but in
>>>    the case they do, a special ClassLoader might contain a reference
>>>    to the authenticated subject, enabling the client to check that a
>>>    proxy's objects unmarshalled into the classloader can be
>>>    authenticated as that subject.
>>>
>>> Typically a Service must authenticate as a Principal, during proxy
>>> verification, the Principal is known prior to authentication and is set
>>> as a
>>> min server principal constraint, the Subject and it's certificate are not
>>> known until after authentication and after class loading.
>>>
>>> There is one more thing to consider. If a classloader namespace is
>>> already
>>> in use for a proxy, and a new proxy is unmarshalled into this
>>> classloader,
>>> that class loader has already been verified by another proxy, during the
>>> period of unmarshalling, the classes in that class loader are trusted,
>>> they're not local code, but they're trusted and permissions have been
>>> granted to a Principal, assuming the code is trustworthy, it should be
>>> safe
>>> to unmarshall unauthenticated proxy objects into this class loader,
>>> because
>>> we trust the code and because if it isn't run as a subject during
>>> unmarshalling, it doesn't gain any privileges.   After unmarshalling if
>>> the
>>> second proxy doesn't authenticate as the same Subject as the first, all
>>> references to it should be set null and the garbage collector allowed to
>>> collect it.  Remember this is the case where proxy's share the same
>>> codebase
>>> URL and Subject.
>>>
>>> In pepe/skunk, I have a backward compatible experimental
>>> MarshalledInstance
>>> that contains a verifier proxy, used to authenticate the original creator
>>> of
>>> the MarshalledInstance, here the Subject could be supplied to something
>>> similar to RMIClassLoader, Gregg has donated his
>>> CodebaseAccessClassloader,
>>> this could have a method added to accept a Subject along with the
>>> annotation, or alternately classloading could be performed using the
>>> Subject.  This authenticated verifier proxy then verifies the codebase
>>> annotation using a message digest.  It is expected that the Subject would
>>> be
>>> the same as that which authenticates the proxy and it's every method call
>>> on
>>> the proxy.  Authentication prior to unmarshalling remote code is intended
>>> to
>>> avoid unmarshalling attacks.
>>>
>>> I plan to experiment, writing some more utility code, based on my proxy
>>> isolate in skunk/pepe, that allows unmarshalling to be performed in an
>>> isolated Thread, which catches a StackOverflowError, in the case of a
>>> remote
>>> end deliberately sending a spuriously large object stream.  Currently
>>> this
>>> only handles the unmarshalling process (MarshalledInstance.get()),
>>> however
>>> I'm looking at how to extend it to deserialization of MarshalledInstance
>>> during discovery.
>>>
>>> But what about Exported objects, not discovered through a lookup service,
>>> but returned from another Service, if these are from a different remote
>>> host
>>> with a different Subject?
>>>
>>> All the above is a thought experiment, feel free to participate and
>>> correct
>>> any errors or fallacies.
>>>
>>> Cheers,
>>>
>>> Peter.
>>>
>>>
>>>
>>>> Cheers,
>>>>
>>>>
>>>>> Peter.
>>>>>
>>>>>  Then of course there's also the argument that we should do nothing,
so
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> consider this a thought experiment to discover how it might be done,
>>>>>> not
>>>>>>
>>>>>>
>>>>>>> that it will or must be, that can be decided later.
>>>>>>>
>>>>>>> What are your thoughts?
>>>>>>>
>>>>>>> Cheers,
>>>>>>>
>>>>>>> Peter.
>>>>>>>
>>>>>>>
>>>>>>> Jeff Ramsdale wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> Chiming in here, perhaps off topic...
>>>>>>>>
>>>>>>>> Sometime back (maybe within the past year?) there seemed
to be
>>>>>>>> agreement on removing the ClassPath manifest attributes moving
>>>>>>>> forward
>>>>>>>> in order to not make assumptions concerning relative jar
locations
>>>>>>>> (e.g. classpath built from local Maven repo).
>>>>>>>>
>>>>>>>> -jeff
>>>>>>>>
>>>>>>>> On Sun, Jan 2, 2011 at 8:36 AM, Greg Trasuk <trasukg@stratuscom.com
>>>>>>>> >
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> On Sun, 2011-01-02 at 11:15, Tom Hobbs wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> Am I right in thinking/remembering that, with the
exception of the
>>>>>>>>>> *-dl.jar files, the only others that are needed are
the jsk-*.jar
>>>>>>>>>> ones.
>>>>>>>>>>
>>>>>>>>>> I'm pretty sure that many of the JARs contain the
same class
>>>>>>>>>> files,
>>>>>>>>>> I
>>>>>>>>>> think that there's definitely scope to reduce the
number JAR files
>>>>>>>>>> that the build creates.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> I think you might be mistaken about that.  The *-dl.jar
files often
>>>>>>>>> contain duplications of classes in *.jar files, but that's
>>>>>>>>> reasonable
>>>>>>>>> and expected.  The few service implementation jar files
that I've
>>>>>>>>> looked
>>>>>>>>> at contain ClassPath manifest attributes that reference
jsk-lib
>>>>>>>>> etc.
>>>>>>>>>
>>>>>>>>> The only real duplication I'm aware of is in the jini-core.jar,
>>>>>>>>> jini-ext.jar and sun-utils.jar files, that duplicate
the contents
>>>>>>>>> of
>>>>>>>>> jsk-platform and jsk-lib.  This is done for backwards
compatability
>>>>>>>>> (that's the way it was in Jini 1.0-days), and could probably
be
>>>>>>>>> deprecated at this point, after consulting with the user
community.
>>>>>>>>>
>>>>>>>>> Cheers,
>>>>>>>>>
>>>>>>>>> Greg.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> On Sun, Jan 2, 2011 at 8:51 AM, Peter Firmstone <jini@zeus.net.au
>>>>>>>>>> >
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> I agree that dynamic proxy classes should remain
dynamic
>>>>>>>>>>> downloads,
>>>>>>>>>>> however
>>>>>>>>>>> much of net.jini.* isn't in the jsk-platform.jar
>>>>>>>>>>>
>>>>>>>>>>> Should we expand the platform to contain all
net.jini.*?
>>>>>>>>>>>
>>>>>>>>>>> Except for providers? (com.sun.jini.resource.Service,
similar to
>>>>>>>>>>> Java's
>>>>>>>>>>> sun.misc.Service and java.util.ServiceLoader)
>>>>>>>>>>>
>>>>>>>>>>> Perhaps we can include more in the platform and
reduce the number
>>>>>>>>>>> of
>>>>>>>>>>> jar
>>>>>>>>>>> archives we've got?
>>>>>>>>>>>
>>>>>>>>>>> Any thoughts?
>>>>>>>>>>>
>>>>>>>>>>> Cheers,
>>>>>>>>>>>
>>>>>>>>>>> Peter.
>>>>>>>>>>>
>>>>>>>>>>> trasukg@trasuk.com wrote:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> Isn't that already jsk-platform.jar?  I would
object to anything
>>>>>>>>>>>> that
>>>>>>>>>>>> subverts the dynamic proxy loading concept
that is central to
>>>>>>>>>>>> Jini.
>>>>>>>>>>>> It is imperative that people don't, for instance,
get the
>>>>>>>>>>>> service-registrar proxy impls in their local
class path.  That
>>>>>>>>>>>> would
>>>>>>>>>>>> break
>>>>>>>>>>>> compatibility with future or alternate impls.
>>>>>>>>>>>>
>>>>>>>>>>>> Cheers,
>>>>>>>>>>>> Greg
>>>>>>>>>>>> ------Original Message------
>>>>>>>>>>>> From: Sim IJskes - QCG
>>>>>>>>>>>> To: river-dev@incubator.apache.org
>>>>>>>>>>>> ReplyTo: river-dev@incubator.apache.org
>>>>>>>>>>>> Subject: river.jar
>>>>>>>>>>>> Sent: Dec 31, 2010 10:07 AM
>>>>>>>>>>>>
>>>>>>>>>>>> Hello,
>>>>>>>>>>>>
>>>>>>>>>>>> anybody have an objection against a river.jar
in the build that
>>>>>>>>>>>> contains
>>>>>>>>>>>> all river runtime classes?
>>>>>>>>>>>>
>>>>>>>>>>>> Gr. Sim
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message