river-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peter Firmstone <j...@zeus.net.au>
Subject Re: [jira] Created: (RIVER-362) Denial of Service during unmarshalling of smart proxy's
Date Wed, 03 Nov 2010 05:37:27 GMT
Michal Kleczek wrote:
> Thanks for taking time to look at this.
> 1. I am open to _any_ refactoring (and changing the code completely) - my code 
> is just a prototype. BTW - would it be possible to put it somewhere in SVN? 

I'll create a place in skunk for you, probably won't  be until the 
weekend though.

> It 
> would be much easier for me to contribute - I don't need a commit access - 
> just a simple way to produce patches. (The version attached to JIRA needs 
> patching badly since it does not work :) )
> 2. I am not sure I understand the compatibility issues. I assume we're talking 
> about a scenario where there is a running service that has MarshalledInstance 
> objects saved somewhere and we want to upgrade it?

Sort of I mean if there's an Registrar using the old version of 
MarshalledInstance,  clients with new versions will be able to 
deserialize MarshalledInstance, but not it's contained object, because 
the stream implementations are different.  Actually I suspect it may be 
able to resolve the class locally, using only the object byte array, not 
too sure, haven't tested it, based on reading the code.

Clients using the earlier version of MarshalledInstance trying to access 
a Registrar that uses a later version wouldn't be able to discover it's 

Reggie's proxy could use Preferred ClassLoading to allow the client to 
use services utilising later versions of MarshalledInstance..

> I think it can be easily solved by making ModuleAnnotatedInputStream capable 
> of reading both String and Module annotations. 

I had some thoughts about making a boot Module part of the serialized 
from of MarshalledInstance, where earlier clients can fall back on the 
String annotation.  Later versions ignore the string annotation if a 
boot Module is present.

> String annotations could be 
> translated to DefaultRmiModule instances installed and used as usual later on.
> I would even say that my version of RMIClassLoaderSpi should be changed so 
> that it can handle both serialized and encoded Modules as well as old style 
> String annotations (lists of URLs). It should also produce old style 
> annotations for DefaultRmiModules - otherwise tools such as VisualVM are not 
> going to work ( tested this :) )
> 3. I found the following code in River dependent on annotations being Strings 
> (and RMIClassLoader as well):
> 1) Anything that makes use of MarshalledObject (for example DiscoveryV1, 
> RemoteEventListener)
> 2) Reggie entries assume annotation is a string
> 3) Outrigger - I don't know the details - just checked it calls 
> RMIClassLoader.getClassAnnotation()
> 4) ProxyTrustVerifier - does a check if a candidate proxy's ClassLoader is a 
> proper RMI ClassLoader by comparing annotations produced by RMIClassLoader
> Looks to me it is not straightforward to remove River dependency on 
> RMIClassLoader (and - to be honest - it was not my goal at all - I wanted to 
> have a piece of code that would plug into existing River)

Gregg Wonderly provided an implementation, it reportedly worked quite 
well, I've yet to reinstate it and run some tests, but it did pass a 
significant number of tests, I'm waiting for trunk to stabilise, then 
I'll play with it again.
> 4. We can easily support scenarios where a client uses old services (the ones 
> not aware of Modules) - see p. 2. But I don't thing there is an easy way for a 
> service that annotates it's objects with Modules to provide a way for an old 
> client to use it.
See suggestion above.

> 5. Didn't thing too much about implementing all this as a URL handler - it 
> looks pretty complicated to me - handling Manifest classpaths, multi URL 
> codebases etc.

Ok, just thought I'd ask the question.

> 6. Placing constraints on Modules is certainly possible but I have no idea how 
> useful it could be and how hard it would be to implement it.
> The only constraint on a Module we need right now (so that the functionality 
> is on-pair with existing River) is Integrity.YES. We handle it by just having 
> a boolean argument in Module methods. 

Did have a look at the MarshalledInstance I attached earlier, it 
contains a serialization verifier proxy, it's possible to have it 
authenticate and throw an IOException during deserialization if it's not 
authorized.  It uses MessageDigests to confirm the byte array's haven't 
changed during transit, it also clones the arrays to guard against 
stolen references.  This could utilise client and server constraints for 
privacy, authentication and minimum server principal.

If deserialization of MarshalledInstance, followed by unmarshalling of 
it's contained object is done isolated within an Executor thread that 
handles StackOverflowError's, then we're about as bullet proof as we can 
get.  The attacker will probably look for OS vulnerabilities instead and 
steal a trusted identity.

If a client has an earlier version of MarshalledInstance, it simply 
deserializes it and discards the proxy, but it still works as expected, 
the client just doesn't have the Denial of Service deserialization and 
codebase download attack protection.

> It made implementation easy (since it is 
> similar to what we have in River right now). Also - I've got no idea how to 
> implement other constraints - we would have do something really smart in 
> PreferredClassLoader.

Getting Class loading right is important to avoid class visibility 
problems.  Because the service impl, proxy and client all share the same 
common Service API, if they're all in child classloaders relative to the 
ServiceAPI, then we won't have any classloading issues, since were using 
common superclass and interface methods to communicate between different 

> 7. I am not following on how you could get rid of CachingProxyTrust since you 
> need to verify two objects.

Perhaps I've confused the issue, I'm referring to annotating the module 
repeatedly with every object in the stream, rather than just once within 
the MarshalledInstance serialized form.



> Thanks,
> Michal
> On Tuesday 02 of November 2010 14:52:57 Peter Firmstone wrote:
>> Thanks Michal, that was helpful, sorry for taking so long to reply.
>> Having chewed the details, I think now it is safe for me to comment, I'm
>> wondering if your open to some refactoring?
>> You've managed to produce a lot of code in a very short time, your
>> productivity is quite impressive and you seem proficient using Secure
>> Jini Services.
>> I've been thinking about your use of objects as annotations, which I
>> readily took to, something that's bothering me is backward
>> compatibility, an Object could be changed to a String, but a String is
>> final and existing implementations are stuck with it. MarshalledInstance
>> is part of the Discovery 2 Protocol, and Reggie, so it's serialized form
>> matters to remote foreign parties.
>> The current way that MarshalledInstance Serialized Form is implemented
>> leaves it open to expansion due to readObject's implementation.  This
>> means we can add fields without breaking backward compatibility.  The
>> earlier implementations can utilise the existing fields and drop any
>> superfluous objects they don't know about.
>> I'm wondering if it's possible to implement a Module URL scheme, similar
>> to Codebase Services?
>> I've also wondered if using an authenticating verifier proxy during
>> deserialization of MarshalledInstance would be enough, saving the need
>> to Cache using the caching proxy trust verifier.  MarshalledInstance is
>> the point of contact for Discovery and Lookup, if we require
>> authentication and verification, then that might be sufficient for
>> MarshalledInstance.
>> We can utilise your Module annotated streams via Jeri, unbeknown to
>> external implementations.
>> I've attached a very rough example draft rudimentary untested
>> MarshalledInstance (TODO: server min principal authentication) which
>> uses defensive copying and message digests to confirm the deserialized
>> state as part of the private implementation of MarshalledInstance.  This
>> makes object integrity the responsibility of the Object itself, rather
>> than an external mechanism.  So in this case privacy is the
>> responsibility of external mechanisms, but object integrity is an
>> internal concern.
>> Gregg Wonderly created a CodebaseAnnotationClassloader (I think that's
>> the name I mentioned earlier), this might remove the need to use
>> RMIClassLoaderSPI and provide the opportunity to add some additional
>> functionality.
>> See below for some more comments:
>> Best Regards,
>> Peter.
>> Michal Kleczek wrote:
>>> On Thursday 28 of October 2010 10:36:38 Peter Firmstone wrote:
>>>> Ok Interesting, anyone implementing a Service should be quite capable of
>>>> implementing equals, we should then specify that equals and hashcode
>>>> methods should be implemented for ProxyTrust.
>>>> For my benefit can you go over the process of how your new code works?
>>>> I'm interested in the choices you've made & why.
>>> Sure.
>>> Several choices were made because:
>>> 1. I wanted to reuse as much as possible from existing River
>>> 2. I wanted to have a working prototype fast :)
>>> Anyway:
>>> 1. The main idea is to have annotations as objects that could be verified
>>> using standard River proxy verification facilities. To be honest the
>>> idea of having Module interface that enables plugging different
>>> classloading mechanisms is something completely optional.
>>> 2. For the client the basic scenario is:
>>> a) The client gets a serialized object
>>> b) When it is deserialized annotations (Modules) are read and "installed"
>>> c) Installing a Module means:
>>> c1) checking if it was already installed
>>> c2) if not - prepare it using VerifyingProxyPreparer (I've choosen not to
>>> place any InvocationConstraints on Modules - I don't think it is
>>> necessary)
>> I'm curious why client MethodConstraints aren't needed?
>>> d) after a Module is installed it is used to load classes
>>> 3. There are several places in River that depend on annotations being
>>> Strings provided by RMIClassLoader. The most important places are
>>> a) Discovery
>>> b) Reggie implementation
>>> c) ProxyTrustVerifier
>>> Since I did not want to modify this code I had to implement
>>> RMIClassLoaderSpi so that it would provide serialized Modules as
>>> Strings. I've choosen to simply serialize them to byte arrays and Base64
>>> encode them.
>>> 4. There are two important Module implementations available:
>>> a) DefaultRmiModule
>>> b) ProxyTrustModule
>>> DefaultRmiModule can be trusted by the client without contacting any
>>> service - it uses RequireDlPermissionClassProvider to load classes.
>>> ProxyTrustModule on the other hand uses a simple PreferredClassProvider
>>> to load classes. It is a smart proxy that implements
>>> getProxyTrustIterator() so that it can be verified by
>>> ProxyTrustVerifier.
>>> On the server before we can annotate our objects with modules we have to
>>> register a Module that will be used as annotation.
>>> If the server does not register any Module - DefaultRmiModule is going to
>>> be used as annotation.
>>> The server can register a ProxyTrustModule as annotation - to do that it
>>> first must export an object that can be contacted to obtain a Module
>>> verifier. In other words - a service must either:
>>> a) export two ProxyTrusts (one for Modules and another one for the
>>> service itself)
>>> b) its ProxyTrust must provide a verifier that is able to verify both a
>>> Module and a service proxy
>>> To make it possible to use my code with existing services without
>>> modifying them I've decided to implement a special ModuleExporter which
>>> will override the service ServerProxyTrust implementation so that
>>> getProxyVerifier will return a Verifier capable of verifying both a
>>> Module and the service proxy. This verifier works as follows:
>>> 1. Check if an object being verified is a Module.
>>> 2. If so - delegate to a module verifier
>>> 3. If not it means we're verifying a service proxy so delegate to a
>>> default verifier.
>>> The problem is though that when this verifier is deserialized the module
>>> that is capable of loading the service proxy verifier is not yet
>>> installed (the client got this verifier to verify a Module before it can
>>> load any classes). That's why I called it LazyCompositeVerifier - the
>>> default verifier is not deserialized until it is actually needed.
>>> The only problem left is that every time a client gets a service proxy it
>>> will issue a remote call twice to get a verifier: first to get a
>>> verifier for a Module and then to get a verifier for a service proxy.
>>> But it is going to be the same verifier!!! So I've decided to implement
>>> a CachingProxyTrust that 1) can be trusted by the client without the
>>> need to issue any remote call (so we need a CachingProxyTrustVerifier
>>> configured on the client)
>>> 2) Will cache a verifier it obtains from its delegate
>>> The problem with CachingProxyTrust is that it has to be used by:
>>> a) a Module
>>> b) a service proxy
>>> That's why we need OverrideProxy - it is returned from ModuleExporter to
>>> the service so that the service is unaware of the CachingProxyTrust but
>>> still can use it to obtain its verifier. To trust OverrideProxy the
>>> client has to have OverrideProxyVerifier configured locally.
>>> I hope I clarified everything a little bit...
>>> Michal

View raw message