river-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peter Firmstone <j...@zeus.net.au>
Subject Re: Fwd: Re: P2P Internet Services - no code downloads, lambda's
Date Wed, 05 Feb 2014 12:56:47 GMT
Thanks Dawid,

Any help is appreciated, I found it's easier to get started by playing 
around something you're familiar with while getting a better 
understanding of a problem you would like to solve.   I'm not the best 
source of advice for getting started, but diving in is a good way to 
figure it out.  I can help with any questions you might have.

Regards,

Peter.


On 5/02/2014 10:40 PM, Dawid Loubser wrote:
> Peter, I just wanted to say that I absolutely love where you are going 
> with this.
> It seems like a blueprint for a thorough modernisation (and consequent 
> increase of relevancy) of River.
> Visionary thinking that is well-articulated. How do we help?
>
> ;-)
>
>
> Op Wo, 2014-02-05 om 22:29 +1000 skryf Peter Firmstone:
>> -------- Original Message --------
>> Subject: 	Re: P2P Internet Services - no code downloads, lambda's
>> Date: 	Wed, 05 Feb 2014 22:28:58 +1000
>> From: 	Peter Firmstone<jini@zeus.net.au  <mailto:jini@zeus.net.au>>
>> To: 	Dennis Reedy<dennis.reedy@gmail.com  <mailto:dennis.reedy@gmail.com>>
>>
>>
>>
>> On 5/02/2014 10:52 AM, Dennis Reedy wrote:
>> >   Hi Peter,
>> >
>> >   I really love your enthusiasm. I'm not sure I understand what reflective proxies
are,
>>
>> Instances of java.lang.reflect.Proxy.
>>
>>
>> >     not sure JVM code generation would not bring in new security issues (not
sure it would not either).
>>
>> There are ways to secure it with minimal complexity for the client, such
>> as using DNS-SRV for discovery and ObjectInputStream over TLS to
>> retrieve the lookup service, no codebase download.
>>
>> The serialization streams at each end can restrict classes to a trusted
>> subset of Java, Jini and objects defined in Service API, and be defined
>> by constraints.
>>
>> Authentication would be required of course, but in this case via a
>> java.lang.reflect.Proxy implementing ProxyTrust and
>> RemoteMethodControl.  Relaxation of restrictions on serialized streams
>> can be performed after authentiication and proxy verification.
>>
>> If the client hasn't authenticated nothing happens.
>>
>> This would be far more secure than standard java Serialization and
>> ObjectStream's
>>
>> Then create a standard interface that any Proxy can implement, to return
>> Entry objects, ServiceID, Service UI's, smart proxy's etc.  To invert
>> the security problem, authenticate before downloading or dynamically
>> generating any code.
>>
>> >     Can you elaborate?
>>
>> For a moment, forget Jini, forget the implementation and wonder, what if.
>>
>> What if Ken, Jim, Peter, Ann, Bob, Brian, Mark and the rest of the gang
>> had technology available to us now, when they were designing Jini?
>>
>> The foundations, theory and benefit of hindsight, without the
>> limitiations of the software, hardware, bandwidth etc...
>>
>>
>> What would the design of a lookup service designed for the internet be
>> like now?
>>
>> What would be its requirements?
>>
>>     1. Minimal bytes transferred across the network, so it remains fast
>>        and responsive?
>>     2. Avoid lost codebase annotations and complex class loading
>>        relationships where possible?
>>     3. Make codebase downloads the exception rather than the norm.
>>     4. Separate the lookup service into different concerns?
>>           1. Registrar - to register services (requires a smart proxy)
>>           2. ServiceLookup - to lookup or be notified of new services
>>              (java.lang.reflect.Proxy is sufficient, no code downloaded).
>>
>> How would you design ServiceLookup?
>>
>>     1. A reflective proxy based service with no code download?  The team
>>        didn't have that choice, reflective proxy's were implemented (by
>>        the same team) later.
>>     2. How might it look if based on Lambda's?
>>
>> Perhaps Client code could use the lookup service like this:
>>
>> ServiceID serviceID = someValue;
>> Class serviceAPI;
>> Collection services = lookupService.lookup( serviceDef->
>> serviceDef.id().equals(serviceID));
>>
>> To retrieve a service with a specific ServiceID.
>>
>> OR
>>
>> Collection services = lookupService.lookup( serviceDef ->
>> serviceDef.types().filter( clazz ->    Proxy.isProxyClass(clazz)&&
>> serviceAPI.isAssignableFrom(clazz) ));
>>
>> To retrieve services that are instances of java.lang.reflect.Proxy and
>> the desired ServiceAPI.
>>
>> Far more complex operations can be performed to filter services by types
>> and attributes, including greater than or older than, Entry comparison,
>> limiting the number of services returned etc, or just about anything
>> else you can think of.  And all of these filters can be executed in
>> parallel at the service registrar using modern concurrent frameworks.
>>
>> The lambda, Class and ServiceID are serialized across the network as
>> method invocations on the Remote lookup service object, the lambda code
>> is dynamically compiled and invoked on the lookup service
>> implementation, and a Collection containing only the matching service is
>> returned in this case.
>>
>> I think lambda's will significantly expand possible service features and
>> capabilities without requiring codebase downloads.
>>
>> // The client implements this using Lambda's
>> interface ServicePredicate extends Predicate<ServiceDefinition>,
>> Serializable {  }
>>
>> // The lookup service implements this, it is not a remote interface.
>> // The client's lambda code calls these methods.
>> interface ServiceDefinition {
>>       ServiceID iD();
>>       Stream<Class>   types();
>>       Stream<Entry>   attributes();
>> }
>>
>> // The lookup service might look like this,
>> // it would be an instance of java.lang.reflect.Proxy
>> // Only ObjectInputStream and ObjectOutputStream's are required,
>> // Marshalled streams are optional.
>> // Serializable classes can be limited to a trusted subset.
>> interface LookupService {
>>       final int TRANSITION_MATCH_NOMATCH = 1<<   0;
>>       final int TRANSITION_NOMATCH_MATCH = 1<<   1;
>>       final int TRANSITION_MATCH_MATCH = 1<<   2;
>>
>>       // lambda's
>>       /* Returns a Collection containing service proxy instances, not
>> ServiceItem's */
>>       default Collection lookup( ServicePredicate sp) throws IOException;
>> //Unfortunately we can't use generics in the returned Collection.
>>       default EventRegistration notify( ServicePredicate sp, int
>> transitions, RemoteEventListener listener, MarshalledObject handback,
>> long leaseDuration) throws IOException;
>>
>>       default ServiceID serviceID() throws IOException;
>>       default String getURI() throws IOException; // public internet
>> location, not the same as getLocator().
>>       default Collection groups() throws IOException;
>> }
>>
>> // The lookup service would now also be a smart proxy that allows
>> service registration
>> // This could be obtained via a java.lang.reflect.Proxy LookupService
>> instance.
>> interface Registrar extends LookupService {
>>       ServiceRegistration register(ServiceItem item, long leaseDuration)
>> throws RemoteException;
>> }
>>
>> // The well loved and worn ServiceRegistrar interface can take advantage
>> of Java 8 default interface methods to preserve backward compatibility.
>> interface ServiceRegistrar extends Registrar {
>>       Object lookup(ServiceTemplate tmpl) throws RemoteException;
>>       ServiceMatches lookup(ServiceTemplate tmpl, int maxMatches) throws
>> RemoteException;
>>       Class[] getEntryClasses(ServiceTemplate tmpl) throws RemoteException;
>>       Object[] getFieldValues(ServiceTemplate tmpl, int setIndex, String
>> field)
>>           throws NoSuchFieldException, RemoteException;
>>       Class[] getServiceTypes(ServiceTemplate tmpl, String prefix)
>>           throws RemoteException;
>>       ServiceID getServiceID();
>>       LookupLocator getLocator() throws RemoteException;
>>       String[] getGroups() throws RemoteException;
>> }
>>
>>
>> >   Being a pragmatic kind of guy, and before we jump into Java 8, perhaps getting
the codebase at least up to Java 7 would be huge. Using development tools applicable for this
decade would be also be a real win IMO. Perhaps:
>> >
>> >   1. Use of generics in service discovery and Configuration (I think you've
already done the latter).
>> +1  But be careful with generics in Service API, can't use<T>   <?>,
only
>> specific like<String>   or<Class>   and even then it must be very
>> carefully considered as we're dealing with runtime unchecked casts.
>>
>> Better to use Collections.checkedCollection where possible on Service
>> API as these represent compile time boundaries, where casts are not checked.
>>
>> As explained in the javadoc:
>>
>>      Returns a dynamically typesafe view of the specified collection. Any
>>      attempt to insert an element of the wrong type will result in an
>>      immediate |ClassCastException|
>>      <http://download.java.net/jdk8/docs/api/java/lang/ClassCastException.html>.
>>      Assuming a collection contains no incorrectly typed elements prior
>>      to the time a dynamically typesafe view is generated, and that all
>>      subsequent access to the collection takes place through the view, it
>>      is /guaranteed/ that the collection cannot contain an incorrectly
>>      typed element. The generics mechanism in the language provides
>>      compile-time (static) type checking, but it is possible to defeat
>>      this mechanism with unchecked casts. Usually this is not a problem,
>>      as the compiler issues warnings on all such unchecked operations.
>>      There are, however, times when static type checking alone is not
>>      sufficient.  For example, suppose a collection is passed to a
>>      third-party library and it is imperative that the library code not
>>      corrupt the collection by inserting an element of the wrong type.
>>
>>      Another use of dynamically typesafe views is debugging. Suppose a
>>      program fails with a |ClassCastException|, indicating that an
>>      incorrectly typed element was put into a parameterized collection.
>>      Unfortunately, the exception can occur at any time after the
>>      erroneous element is inserted, so it typically provides little or no
>>      information as to the real source of the problem. If the problem is
>>      reproducible, one can quickly determine its source by temporarily
>>      modifying the program to wrap the collection with a dynamically
>>      typesafe view.
>>
>>
>> >   2. Use of annotations in services for lifecycle and injection of required
services and configuration properties.
>> ++1.
>> >   3. River and other technologies (Spring, JEE, OSGi), how to integrate and
use.
>> +1 requires the ClassLoading provider, based on works by Gregg and Sim.
>>
>> >   4. Examples of how to create a project using up to date build tools (Maven,
Gradle, Grunt?).
>> +1
>> >   5. Get rid of classdepandjar. Adopt conventions that use a project structure
representative of the architectural role of each jar file created.
>> +1.  See RIO website, for anyone who's interested in these conventions.
>> >   6. Examples of how to test your services using JUnit and/or TestNG, including
bootstrapping core River services needed for integration testing.
>> +1.
>> >   7. River in the cloud (a River AMI might be a start).
>> More info?
>> >   8. River and big data technologies, applicable for this area?
>> +1.
>> >   9. Seriously address what River includes as services; do we need Norm, Phoenix,
Fiddler?
>> +1 separate out as sepate builds on different development time schedules.
>> >   10. River and other JVM languages (Scala, Clojure)
>> >
>> >   There is lots of work (adding to all the work you've already done in qa_refactor)
just to make River understandable (in a use-case point of view) for developers to grasp how
it can be used to even augment their existing systems (forget about a total green field project
that uses River from the get-go). Just modernizing River is a tall effort.
>> >
>> >   Additionally, now we have the Internet of Things (IoT) and the Alljoyn effort
(https://www.alljoyn.org), how does that fit (if at all) into what River can do? Do we care?
>>
>> Note sure, don't know anything about it.
>> >   The other way to think about your idea is to do it outside of River, might
actually be easier.
>>
>> Yes, quite possibly.
>>
>> >   Just my $0.02
>> >
>> >   Regards
>> >
>> >   Dennis
>> >
>> >   On Feb 4, 2014, at 652PM, Peter<jini@zeus.net.au  <mailto:jini@zeus.net.au>>
   wrote:
>> >
>> >>   Components:
>> >>
>> >>   DNS-SRV Discovery
>> >>   UDT JERI Endpoints- Firewall traversal&    superior WAN performance
>> >>   TLS Encryption
>> >>   Reflective Proxies - no code downloads
>> >>   Lambda expressions (for services) - remote dynamic code generation (this
is big, don't download a codebase, the jvm generates code on demand)
>> >>   Secure serialization - limit classes to java.lang, and Jini trusted subsets.
>> >>   Size constraints on method invocation returns - anti dos measure
>> >>
>> >>   A new internet lookup service, reflective proxy, with lambda expression
based lookup.
>> >>
>> >>
>> >>   There's not a great deal of work required to do this and it should make
a huge difference to our userbase, which at present appears limited.
>> >>
>> >>   Thoughts?
>> >>
>> >>   Peter.
>>
>>


Mime
View raw message