river-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dawid Loubser <da...@travellinck.com>
Subject Re: Fwd: Re: P2P Internet Services - no code downloads, lambda's
Date Wed, 05 Feb 2014 12:40:27 GMT
Peter, I just wanted to say that I absolutely love where you are going
with this.
It seems like a blueprint for a thorough modernisation (and consequent
increase of relevancy) of River.
Visionary thinking that is well-articulated. How do we help?

;-)


Op Wo, 2014-02-05 om 22:29 +1000 skryf Peter Firmstone:

> 
> -------- Original Message --------
> Subject: 	Re: P2P Internet Services - no code downloads, lambda's
> Date: 	Wed, 05 Feb 2014 22:28:58 +1000
> From: 	Peter Firmstone <jini@zeus.net.au>
> To: 	Dennis Reedy <dennis.reedy@gmail.com>
> 
> 
> 
> On 5/02/2014 10:52 AM, Dennis Reedy wrote:
> >  Hi Peter,
> >
> >  I really love your enthusiasm. I'm not sure I understand what reflective proxies
are,
> 
> Instances of java.lang.reflect.Proxy.
> 
> 
> >    not sure JVM code generation would not bring in new security issues (not sure
it would not either).
> 
> There are ways to secure it with minimal complexity for the client, such
> as using DNS-SRV for discovery and ObjectInputStream over TLS to
> retrieve the lookup service, no codebase download.
> 
> The serialization streams at each end can restrict classes to a trusted
> subset of Java, Jini and objects defined in Service API, and be defined
> by constraints.
> 
> Authentication would be required of course, but in this case via a
> java.lang.reflect.Proxy implementing ProxyTrust and
> RemoteMethodControl.  Relaxation of restrictions on serialized streams
> can be performed after authentiication and proxy verification.
> 
> If the client hasn't authenticated nothing happens.
> 
> This would be far more secure than standard java Serialization and
> ObjectStream's
> 
> Then create a standard interface that any Proxy can implement, to return
> Entry objects, ServiceID, Service UI's, smart proxy's etc.  To invert
> the security problem, authenticate before downloading or dynamically
> generating any code.
> 
> >    Can you elaborate?
> 
> For a moment, forget Jini, forget the implementation and wonder, what if.
> 
> What if Ken, Jim, Peter, Ann, Bob, Brian, Mark and the rest of the gang
> had technology available to us now, when they were designing Jini?
> 
> The foundations, theory and benefit of hindsight, without the
> limitiations of the software, hardware, bandwidth etc...
> 
> 
> What would the design of a lookup service designed for the internet be
> like now?
> 
> What would be its requirements?
> 
>    1. Minimal bytes transferred across the network, so it remains fast
>       and responsive?
>    2. Avoid lost codebase annotations and complex class loading
>       relationships where possible?
>    3. Make codebase downloads the exception rather than the norm.
>    4. Separate the lookup service into different concerns?
>          1. Registrar - to register services (requires a smart proxy)
>          2. ServiceLookup - to lookup or be notified of new services
>             (java.lang.reflect.Proxy is sufficient, no code downloaded).
> 
> How would you design ServiceLookup?
> 
>    1. A reflective proxy based service with no code download?  The team
>       didn't have that choice, reflective proxy's were implemented (by
>       the same team) later.
>    2. How might it look if based on Lambda's?
> 
> Perhaps Client code could use the lookup service like this:
> 
> ServiceID serviceID = someValue;
> Class serviceAPI;
> Collection services = lookupService.lookup( serviceDef->
> serviceDef.id().equals(serviceID));
> 
> To retrieve a service with a specific ServiceID.
> 
> OR
> 
> Collection services = lookupService.lookup( serviceDef ->
> serviceDef.types().filter( clazz ->   Proxy.isProxyClass(clazz)&&
> serviceAPI.isAssignableFrom(clazz) ));
> 
> To retrieve services that are instances of java.lang.reflect.Proxy and
> the desired ServiceAPI.
> 
> Far more complex operations can be performed to filter services by types
> and attributes, including greater than or older than, Entry comparison,
> limiting the number of services returned etc, or just about anything
> else you can think of.  And all of these filters can be executed in
> parallel at the service registrar using modern concurrent frameworks.
> 
> The lambda, Class and ServiceID are serialized across the network as
> method invocations on the Remote lookup service object, the lambda code
> is dynamically compiled and invoked on the lookup service
> implementation, and a Collection containing only the matching service is
> returned in this case.
> 
> I think lambda's will significantly expand possible service features and
> capabilities without requiring codebase downloads.
> 
> // The client implements this using Lambda's
> interface ServicePredicate extends Predicate<ServiceDefinition>,
> Serializable {  }
> 
> // The lookup service implements this, it is not a remote interface.
> // The client's lambda code calls these methods.
> interface ServiceDefinition {
>      ServiceID iD();
>      Stream<Class>  types();
>      Stream<Entry>  attributes();
> }
> 
> // The lookup service might look like this,
> // it would be an instance of java.lang.reflect.Proxy
> // Only ObjectInputStream and ObjectOutputStream's are required,
> // Marshalled streams are optional.
> // Serializable classes can be limited to a trusted subset.
> interface LookupService {
>      final int TRANSITION_MATCH_NOMATCH = 1<<  0;
>      final int TRANSITION_NOMATCH_MATCH = 1<<  1;
>      final int TRANSITION_MATCH_MATCH = 1<<  2;
> 
>      // lambda's
>      /* Returns a Collection containing service proxy instances, not
> ServiceItem's */
>      default Collection lookup( ServicePredicate sp) throws IOException;
> //Unfortunately we can't use generics in the returned Collection.
>      default EventRegistration notify( ServicePredicate sp, int
> transitions, RemoteEventListener listener, MarshalledObject handback,
> long leaseDuration) throws IOException;
> 
>      default ServiceID serviceID() throws IOException;
>      default String getURI() throws IOException; // public internet
> location, not the same as getLocator().
>      default Collection groups() throws IOException;
> }
> 
> // The lookup service would now also be a smart proxy that allows
> service registration
> // This could be obtained via a java.lang.reflect.Proxy LookupService
> instance.
> interface Registrar extends LookupService {
>      ServiceRegistration register(ServiceItem item, long leaseDuration)
> throws RemoteException;
> }
> 
> // The well loved and worn ServiceRegistrar interface can take advantage
> of Java 8 default interface methods to preserve backward compatibility.
> interface ServiceRegistrar extends Registrar {
>      Object lookup(ServiceTemplate tmpl) throws RemoteException;
>      ServiceMatches lookup(ServiceTemplate tmpl, int maxMatches) throws
> RemoteException;
>      Class[] getEntryClasses(ServiceTemplate tmpl) throws RemoteException;
>      Object[] getFieldValues(ServiceTemplate tmpl, int setIndex, String
> field)
>          throws NoSuchFieldException, RemoteException;
>      Class[] getServiceTypes(ServiceTemplate tmpl, String prefix)
>          throws RemoteException;
>      ServiceID getServiceID();
>      LookupLocator getLocator() throws RemoteException;
>      String[] getGroups() throws RemoteException;
> }
> 
> 
> >  Being a pragmatic kind of guy, and before we jump into Java 8, perhaps getting
the codebase at least up to Java 7 would be huge. Using development tools applicable for this
decade would be also be a real win IMO. Perhaps:
> >
> >  1. Use of generics in service discovery and Configuration (I think you've already
done the latter).
> +1  But be careful with generics in Service API, can't use<T>  <?>, only
> specific like<String>  or<Class>  and even then it must be very
> carefully considered as we're dealing with runtime unchecked casts.
> 
> Better to use Collections.checkedCollection where possible on Service
> API as these represent compile time boundaries, where casts are not checked.
> 
> As explained in the javadoc:
> 
>     Returns a dynamically typesafe view of the specified collection. Any
>     attempt to insert an element of the wrong type will result in an
>     immediate |ClassCastException|
>     <http://download.java.net/jdk8/docs/api/java/lang/ClassCastException.html>.
>     Assuming a collection contains no incorrectly typed elements prior
>     to the time a dynamically typesafe view is generated, and that all
>     subsequent access to the collection takes place through the view, it
>     is /guaranteed/ that the collection cannot contain an incorrectly
>     typed element. The generics mechanism in the language provides
>     compile-time (static) type checking, but it is possible to defeat
>     this mechanism with unchecked casts. Usually this is not a problem,
>     as the compiler issues warnings on all such unchecked operations.
>     There are, however, times when static type checking alone is not
>     sufficient.  For example, suppose a collection is passed to a
>     third-party library and it is imperative that the library code not
>     corrupt the collection by inserting an element of the wrong type.
> 
>     Another use of dynamically typesafe views is debugging. Suppose a
>     program fails with a |ClassCastException|, indicating that an
>     incorrectly typed element was put into a parameterized collection.
>     Unfortunately, the exception can occur at any time after the
>     erroneous element is inserted, so it typically provides little or no
>     information as to the real source of the problem. If the problem is
>     reproducible, one can quickly determine its source by temporarily
>     modifying the program to wrap the collection with a dynamically
>     typesafe view.
> 
> 
> >  2. Use of annotations in services for lifecycle and injection of required services
and configuration properties.
> ++1.
> >  3. River and other technologies (Spring, JEE, OSGi), how to integrate and use.
> +1 requires the ClassLoading provider, based on works by Gregg and Sim.
> 
> >  4. Examples of how to create a project using up to date build tools (Maven, Gradle,
Grunt?).
> +1
> >  5. Get rid of classdepandjar. Adopt conventions that use a project structure representative
of the architectural role of each jar file created.
> +1.  See RIO website, for anyone who's interested in these conventions.
> >  6. Examples of how to test your services using JUnit and/or TestNG, including bootstrapping
core River services needed for integration testing.
> +1.
> >  7. River in the cloud (a River AMI might be a start).
> More info?
> >  8. River and big data technologies, applicable for this area?
> +1.
> >  9. Seriously address what River includes as services; do we need Norm, Phoenix,
Fiddler?
> +1 separate out as sepate builds on different development time schedules.
> >  10. River and other JVM languages (Scala, Clojure)
> >
> >  There is lots of work (adding to all the work you've already done in qa_refactor)
just to make River understandable (in a use-case point of view) for developers to grasp how
it can be used to even augment their existing systems (forget about a total green field project
that uses River from the get-go). Just modernizing River is a tall effort.
> >
> >  Additionally, now we have the Internet of Things (IoT) and the Alljoyn effort (https://www.alljoyn.org),
how does that fit (if at all) into what River can do? Do we care?
> 
> Note sure, don't know anything about it.
> >  The other way to think about your idea is to do it outside of River, might actually
be easier.
> 
> Yes, quite possibly.
> 
> >  Just my $0.02
> >
> >  Regards
> >
> >  Dennis
> >
> >  On Feb 4, 2014, at 652PM, Peter<jini@zeus.net.au>   wrote:
> >
> >>  Components:
> >>
> >>  DNS-SRV Discovery
> >>  UDT JERI Endpoints- Firewall traversal&   superior WAN performance
> >>  TLS Encryption
> >>  Reflective Proxies - no code downloads
> >>  Lambda expressions (for services) - remote dynamic code generation (this is
big, don't download a codebase, the jvm generates code on demand)
> >>  Secure serialization - limit classes to java.lang, and Jini trusted subsets.
> >>  Size constraints on method invocation returns - anti dos measure
> >>
> >>  A new internet lookup service, reflective proxy, with lambda expression based
lookup.
> >>
> >>
> >>  There's not a great deal of work required to do this and it should make a huge
difference to our userbase, which at present appears limited.
> >>
> >>  Thoughts?
> >>
> >>  Peter.
> 
> 

Mime
View raw message