river-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peter Firmstone <j...@zeus.net.au>
Subject Fwd: Re: P2P Internet Services - no code downloads, lambda's
Date Wed, 05 Feb 2014 12:29:51 GMT


-------- Original Message --------
Subject: 	Re: P2P Internet Services - no code downloads, lambda's
Date: 	Wed, 05 Feb 2014 22:28:58 +1000
From: 	Peter Firmstone <jini@zeus.net.au>
To: 	Dennis Reedy <dennis.reedy@gmail.com>



On 5/02/2014 10:52 AM, Dennis Reedy wrote:
>  Hi Peter,
>
>  I really love your enthusiasm. I'm not sure I understand what reflective proxies are,

Instances of java.lang.reflect.Proxy.


>    not sure JVM code generation would not bring in new security issues (not sure it would
not either).

There are ways to secure it with minimal complexity for the client, such
as using DNS-SRV for discovery and ObjectInputStream over TLS to
retrieve the lookup service, no codebase download.

The serialization streams at each end can restrict classes to a trusted
subset of Java, Jini and objects defined in Service API, and be defined
by constraints.

Authentication would be required of course, but in this case via a
java.lang.reflect.Proxy implementing ProxyTrust and
RemoteMethodControl.  Relaxation of restrictions on serialized streams
can be performed after authentiication and proxy verification.

If the client hasn't authenticated nothing happens.

This would be far more secure than standard java Serialization and
ObjectStream's

Then create a standard interface that any Proxy can implement, to return
Entry objects, ServiceID, Service UI's, smart proxy's etc.  To invert
the security problem, authenticate before downloading or dynamically
generating any code.

>    Can you elaborate?

For a moment, forget Jini, forget the implementation and wonder, what if.

What if Ken, Jim, Peter, Ann, Bob, Brian, Mark and the rest of the gang
had technology available to us now, when they were designing Jini?

The foundations, theory and benefit of hindsight, without the
limitiations of the software, hardware, bandwidth etc...


What would the design of a lookup service designed for the internet be
like now?

What would be its requirements?

   1. Minimal bytes transferred across the network, so it remains fast
      and responsive?
   2. Avoid lost codebase annotations and complex class loading
      relationships where possible?
   3. Make codebase downloads the exception rather than the norm.
   4. Separate the lookup service into different concerns?
         1. Registrar - to register services (requires a smart proxy)
         2. ServiceLookup - to lookup or be notified of new services
            (java.lang.reflect.Proxy is sufficient, no code downloaded).

How would you design ServiceLookup?

   1. A reflective proxy based service with no code download?  The team
      didn't have that choice, reflective proxy's were implemented (by
      the same team) later.
   2. How might it look if based on Lambda's?

Perhaps Client code could use the lookup service like this:

ServiceID serviceID = someValue;
Class serviceAPI;
Collection services = lookupService.lookup( serviceDef->
serviceDef.id().equals(serviceID));

To retrieve a service with a specific ServiceID.

OR

Collection services = lookupService.lookup( serviceDef ->
serviceDef.types().filter( clazz ->   Proxy.isProxyClass(clazz)&&
serviceAPI.isAssignableFrom(clazz) ));

To retrieve services that are instances of java.lang.reflect.Proxy and
the desired ServiceAPI.

Far more complex operations can be performed to filter services by types
and attributes, including greater than or older than, Entry comparison,
limiting the number of services returned etc, or just about anything
else you can think of.  And all of these filters can be executed in
parallel at the service registrar using modern concurrent frameworks.

The lambda, Class and ServiceID are serialized across the network as
method invocations on the Remote lookup service object, the lambda code
is dynamically compiled and invoked on the lookup service
implementation, and a Collection containing only the matching service is
returned in this case.

I think lambda's will significantly expand possible service features and
capabilities without requiring codebase downloads.

// The client implements this using Lambda's
interface ServicePredicate extends Predicate<ServiceDefinition>,
Serializable {  }

// The lookup service implements this, it is not a remote interface.
// The client's lambda code calls these methods.
interface ServiceDefinition {
     ServiceID iD();
     Stream<Class>  types();
     Stream<Entry>  attributes();
}

// The lookup service might look like this,
// it would be an instance of java.lang.reflect.Proxy
// Only ObjectInputStream and ObjectOutputStream's are required,
// Marshalled streams are optional.
// Serializable classes can be limited to a trusted subset.
interface LookupService {
     final int TRANSITION_MATCH_NOMATCH = 1<<  0;
     final int TRANSITION_NOMATCH_MATCH = 1<<  1;
     final int TRANSITION_MATCH_MATCH = 1<<  2;

     // lambda's
     /* Returns a Collection containing service proxy instances, not
ServiceItem's */
     default Collection lookup( ServicePredicate sp) throws IOException;
//Unfortunately we can't use generics in the returned Collection.
     default EventRegistration notify( ServicePredicate sp, int
transitions, RemoteEventListener listener, MarshalledObject handback,
long leaseDuration) throws IOException;

     default ServiceID serviceID() throws IOException;
     default String getURI() throws IOException; // public internet
location, not the same as getLocator().
     default Collection groups() throws IOException;
}

// The lookup service would now also be a smart proxy that allows
service registration
// This could be obtained via a java.lang.reflect.Proxy LookupService
instance.
interface Registrar extends LookupService {
     ServiceRegistration register(ServiceItem item, long leaseDuration)
throws RemoteException;
}

// The well loved and worn ServiceRegistrar interface can take advantage
of Java 8 default interface methods to preserve backward compatibility.
interface ServiceRegistrar extends Registrar {
     Object lookup(ServiceTemplate tmpl) throws RemoteException;
     ServiceMatches lookup(ServiceTemplate tmpl, int maxMatches) throws
RemoteException;
     Class[] getEntryClasses(ServiceTemplate tmpl) throws RemoteException;
     Object[] getFieldValues(ServiceTemplate tmpl, int setIndex, String
field)
         throws NoSuchFieldException, RemoteException;
     Class[] getServiceTypes(ServiceTemplate tmpl, String prefix)
         throws RemoteException;
     ServiceID getServiceID();
     LookupLocator getLocator() throws RemoteException;
     String[] getGroups() throws RemoteException;
}


>  Being a pragmatic kind of guy, and before we jump into Java 8, perhaps getting the codebase
at least up to Java 7 would be huge. Using development tools applicable for this decade would
be also be a real win IMO. Perhaps:
>
>  1. Use of generics in service discovery and Configuration (I think you've already done
the latter).
+1  But be careful with generics in Service API, can't use<T>  <?>, only
specific like<String>  or<Class>  and even then it must be very
carefully considered as we're dealing with runtime unchecked casts.

Better to use Collections.checkedCollection where possible on Service
API as these represent compile time boundaries, where casts are not checked.

As explained in the javadoc:

    Returns a dynamically typesafe view of the specified collection. Any
    attempt to insert an element of the wrong type will result in an
    immediate |ClassCastException|
    <http://download.java.net/jdk8/docs/api/java/lang/ClassCastException.html>.
    Assuming a collection contains no incorrectly typed elements prior
    to the time a dynamically typesafe view is generated, and that all
    subsequent access to the collection takes place through the view, it
    is /guaranteed/ that the collection cannot contain an incorrectly
    typed element. The generics mechanism in the language provides
    compile-time (static) type checking, but it is possible to defeat
    this mechanism with unchecked casts. Usually this is not a problem,
    as the compiler issues warnings on all such unchecked operations.
    There are, however, times when static type checking alone is not
    sufficient.  For example, suppose a collection is passed to a
    third-party library and it is imperative that the library code not
    corrupt the collection by inserting an element of the wrong type.

    Another use of dynamically typesafe views is debugging. Suppose a
    program fails with a |ClassCastException|, indicating that an
    incorrectly typed element was put into a parameterized collection.
    Unfortunately, the exception can occur at any time after the
    erroneous element is inserted, so it typically provides little or no
    information as to the real source of the problem. If the problem is
    reproducible, one can quickly determine its source by temporarily
    modifying the program to wrap the collection with a dynamically
    typesafe view.


>  2. Use of annotations in services for lifecycle and injection of required services and
configuration properties.
++1.
>  3. River and other technologies (Spring, JEE, OSGi), how to integrate and use.
+1 requires the ClassLoading provider, based on works by Gregg and Sim.

>  4. Examples of how to create a project using up to date build tools (Maven, Gradle,
Grunt?).
+1
>  5. Get rid of classdepandjar. Adopt conventions that use a project structure representative
of the architectural role of each jar file created.
+1.  See RIO website, for anyone who's interested in these conventions.
>  6. Examples of how to test your services using JUnit and/or TestNG, including bootstrapping
core River services needed for integration testing.
+1.
>  7. River in the cloud (a River AMI might be a start).
More info?
>  8. River and big data technologies, applicable for this area?
+1.
>  9. Seriously address what River includes as services; do we need Norm, Phoenix, Fiddler?
+1 separate out as sepate builds on different development time schedules.
>  10. River and other JVM languages (Scala, Clojure)
>
>  There is lots of work (adding to all the work you've already done in qa_refactor) just
to make River understandable (in a use-case point of view) for developers to grasp how it
can be used to even augment their existing systems (forget about a total green field project
that uses River from the get-go). Just modernizing River is a tall effort.
>
>  Additionally, now we have the Internet of Things (IoT) and the Alljoyn effort (https://www.alljoyn.org),
how does that fit (if at all) into what River can do? Do we care?

Note sure, don't know anything about it.
>  The other way to think about your idea is to do it outside of River, might actually
be easier.

Yes, quite possibly.

>  Just my $0.02
>
>  Regards
>
>  Dennis
>
>  On Feb 4, 2014, at 652PM, Peter<jini@zeus.net.au>   wrote:
>
>>  Components:
>>
>>  DNS-SRV Discovery
>>  UDT JERI Endpoints- Firewall traversal&   superior WAN performance
>>  TLS Encryption
>>  Reflective Proxies - no code downloads
>>  Lambda expressions (for services) - remote dynamic code generation (this is big,
don't download a codebase, the jvm generates code on demand)
>>  Secure serialization - limit classes to java.lang, and Jini trusted subsets.
>>  Size constraints on method invocation returns - anti dos measure
>>
>>  A new internet lookup service, reflective proxy, with lambda expression based lookup.
>>
>>
>>  There's not a great deal of work required to do this and it should make a huge difference
to our userbase, which at present appears limited.
>>
>>  Thoughts?
>>
>>  Peter.



Mime
View raw message