river-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bishnu Gautam <bishn...@hotmail.com>
Subject RE: Lotj - languages other than java
Date Wed, 06 Jul 2016 10:11:49 GMT
Thanks Peter For the introduction of new implementation. I will go through it.Between, here
is the scenario that I often faced and I am pretty sure that most of the users have faced
is this:
1. It is possible to invoke lookup service from the internet and even getting the registrar
is possible. In fact I was able to find the lookup server behind the NAT and firewall.2. Once
I get the registrar the service registered in this registrar seems to have local IP3. Therefore
the client from the internet is not able to set up the connection by using the proxy as it
gave you local IP.
This problem occurred because the lookup service was running in private IP and not assigned
by global IP directly into its interface. If the lookupserver is directly assigned by global
IP, there would be no problem to publicize the current lookup server. However, in our case
and most of the servers inside the networks are not directly assigned global IP but are assigned
in a gateway machine having NAT facility due to security reasons. The static NAT  and some
other techniques are applied in order to access the services through outside network and it
is quite possible because they are port forwarded and synchronize the filtering rules properly.
However port forwarding technique alone is not helpful in the case of river. Because registrar
(kept inside firewall and NAT) seems to have the proxy service which has local IP whenever
it is downloaded to the client machine. In the first connection, it is possible to locate
the lookuplocater by using port forwarding technique however once it is located it returns
the proxy services having local IP. This scenario is becoming a possible design flaw  in River
and limiting the users of this technology.I would like to suggest you to assign domain name
in any case rather than assigning IP in its proxy. Because if you assign domain name rather
than IP, the lookup can be made without considering whether a call from local network or global
network. Because when lookup is done from within a network, dns will return local IP and it
will return global IP whenever it is done from outside network. In this case, there will be
no problem once it is accessed by port forwarding technique too. So, we should rely upon DNS.
So, it is quite possible that the codebase is assigning the local IP to its proxy service.
So, need to tweak this scenario too. This seems lookup overhead but it seems to be the only
solution to overcome NAT and Firewall locking to the River solution. I think that this issue
can be solved by you guys with minimal effort. Let me know if you could tweak the code of
registrar as explained above. This will certainly enable a huge potential to this technology.
Please correct me if something is missing here.

Bishnu Prasad Gautam

> Date: Wed, 6 Jul 2016 16:25:12 +1000
> From: jini@zeus.net.au
> To: dev@river.apache.org
> Subject: Re: Lotj - languages other than java
> Thanks Bishnu,
> I plan to set up a public lookup service later this year, or early next 
> over IPv6 (dependant on me completing the work on github), there's a 
> beta release of the code available you can experiment with.
> It will be possible to discover this lookup service dynamically using 
> the global IPv6 multicast address FF0X::155 for Jini-announcement.
> For DOS reasons, Jini-request will only be available for local network use.
> http://www.iana.org/assignments/ipv6-multicast-addresses/ipv6-multicast-addresses.xhtml
> Before I could make a lookup service publicly available, I've had to do 
> a lot of work on addressing security issues:
>    1. Serialization atomic input validation.
>    2. Two new default methods in ServiceRegistrar so that clients can
>       authenticate services, prior to deserializing their proxy's,
>       Entry's and downloading or provisioning codebases.
>    3. New discovery providers with updated cryptographic hash functions
>       for secure discovery and added input validation for
>       deserialization of registrar proxy.
>    4. Dynamic granting of DownloadPermission and
>       DeserializationPermission (similar to deserialization
>       whitelisting) to authenticated services (to allow their proxy to
>       be deserialized / downloaded).
>    5. Allow listing of Permissions within jar files, for smart proxy's
>       and have these granted dynamically if service authentication succeeds.
>    6. Update encryption cyphers and invocation constraints.
> Existing River / Jini deployments will not be able to contact this 
> lookup service, in the event they could, they cannot satisfy the 
> ConfidentialityStrength invocation constraints either unfortunately.  On 
> local networks security could be relaxed sufficiently to allow 
> compatibility with an existing River deployment, but public lookup 
> services should not make that compromise.
> The Serial form of objects are compatible with River.
> This is all beta, new API may change if adopted by River.
> Regards,
> Peter.
> On 6/07/2016 3:36 PM, Bishnu Gautam wrote:
> > Thanks Peter for your detail explanation regarding River on Internet.I will go through
it. By the way do you have any publicly available Lookup service to which I can register my
proxy service and this proxy service will refer my web server by which I could download my
rest of the actual service codes (fully implemented) from the publicly available web server(in
which my service codes are located). In fact I am trying to avail Lookup Server publicly so
that the user could register their proxy services in my lookup server.Let me know if anybody
already made it available for public use. Once we are able to do it we do not need to rely
upon REST too and also do not need to tackle with NATing and packet filtering issues. I am
looking whether there are any workable publicly available lookup services to which I can register
my proxy.
> > RegardsBishnu
> >
> >
> > Bishnu Prasad Gautam
> >
> >
> >> Date: Tue, 5 Jul 2016 17:46:14 +1000
> >> From: jini@zeus.net.au
> >> To: dev@river.apache.org
> >> Subject: Re: Lotj - languages other than java
> >>
> >> Thanks Bishnu,
> >>
> >> Mark Brouwer originally pointed out many years ago, that while jini had
> >> https jeri endpoints, there was no support to perform unicast discover
> >> over https in LookupLocator discovery.
> >>
> >> I have implemented that.
> >>
> >> Now, you can have any number of publicly available lookup services, and
> >> contact them using https unicast with LookupLocator,
> >> ConstrainableLookupLocator and LookupLocatorDiscovery.  DNS-SD is an
> >> obvious substitute for multicast discovery for clients behind firewalls,
> >> but it hasn't been implemented yet.
> >>
> >> With public lookup services, even when not discoverable with multicast
> >> discovery (because your clients are behind firewalls), can register with
> >> each other, using multicast discovery, allowing clients behind firewalls
> >> to find others, so they only need to know a few permanent registrars to
> >> find others, when they don't have access to multicast discovery.
> >>
> >> Non public service clients (behind firewalls or NAT) can locate public
> >> services.  Non public clients can listen behind NAT and firewalls, when
> >> they register a listener with a lookup service, they will be notified of
> >> new service registrations, because https jeri endpoints will keep the
> >> connection (ports) open between a lookup service and clients behind
> >> firewalls.
> >>
> >> So lets say for example, you have an embedded client behind a firewall,
> >> which is also a master for an IoT wireless local network and there are
> >> several devices that will send a video stream or other data, then for
> >> instance a public consumer service, could register with a lookup service
> >> where the embedded client was listening, then be contacted and
> >> authenticated by the embedded client and access a service directly from
> >> the client to receive live data streams from that wireless network, this
> >> may then republish the data accumulated from multiple such embedded
> >> clients via a web site.  Clients behind NAT must initiate contact, only
> >> then can they be contacted.
> >>
> >> JERI multiplexes, so you can have 127 active service connections over
> >> one connection between two nodes.
> >>
> >> LetsEncrypt.org is a free certificate authority than can be utilised for
> >> https jeri endpoints.  LetsEncrypt.org doesn't provide code
> >> certificates, so they can't be used for signing jar files.
> >>
> >> To avoid the expense of CA signed codebase certificates, https discovery
> >> automatically grants DownloadPermission and DeserializationPermission
> >> (which is required to allow proxy deserialization over a https jeri
> >> endpoint with an InputValidation constraint), to an anoymous code signer
> >> certificate and uri exchanged during the discovery process, but only if
> >> authentication is successful.  The codebase jar file can also contain
> >> the permissions it requires, so that these can be granted dynamically by
> >> the client during proxy preparation.
> >>
> >> In addition https jeri endpoint encryption cyphers have all been updated
> >> to modern secure cyphers and support for insecure cyphers has been removed.
> >>
> >> Default methods have been added to ServiceRegistrar, to allow the client
> >> to authenticate services prior to retrieving proxy's and Entry's.
> >> ProxyTrust verfiiers are no longer necessary as the proxy is obtained
> >> directly from the service after authentication, rather than via a third
> >> party.  The proxy is already trusted.
> >>
> >> https://pfirmstone.github.io/river-internet/
> >>
> >> I hope to get this accepted by River at some point, I figure that
> >> creating a demonstration will assist the understanding process for other
> >> developers, as I wasn't able to communicate effectively enough to avoid
> >> strong resistance and criticism when I originally proposed it.
> >>
> >> Regards,
> >>
> >> Peter.
> >>
> >>
> >> On 5/07/2016 1:33 PM, Bishnu Gautam wrote:
> >>> Hi Peter
> >>> It is great that you pointed out lookup locator issue in firewall and its
potential solution. It would be great to see the developments in River in which they really
focus to have lookup discovery beyond the firewall without requiring port forward and other
demanding packet filtering techniques. Once this obstacle in River is crossed, I am pretty
sure that there will be new paradigm shift in IoT or ICT technology. This technology has a
tremendous potential. However, I never understand why River community never try to bring this
issue on the first place. River in Internet would be the most wonderful solutions for millions
of users around the world. Please think, discuss and try to work on it. It would be a great
news for us.
> >>> RegardsBishnu
> >>>
> >>>
> >>> Bishnu Prasad Gautam
> >>>
> >>>
> >>>> Date: Mon, 4 Jul 2016 18:37:25 +1000
> >>>> From: jini@zeus.net.au
> >>>> Subject: Re: Lotj - languages other than java
> >>>> To: dev@river.apache.org
> >>>> CC: simon@qcg.nl
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> Sim,
> >>>>
> >>>> I'd like to see the project return to the days where we had a number
of active committers working together on the same goals.
> >>>>
> >>>> I've got a project on github, where I've continued work that was controversial,
I'd like to bring this work back to the project if possible.  It has some minor breaking changes,
but if backward compatibility was essential, it could be accommodated.
> >>>>
> >>>> Changes:
> >>>> * New secure discovery providers, including https among others, including
added https protocol support for LookupLocator.  However since firewalls may not allow ipv6
multicast packets through, DNS-SD would be useful.
> >>>> * IPv6 Discovery, global and local groups.
> >>>> * Discovery V2 support added to LookupLocator's getRegistrar method.
> >>>> * Updated encryption ciphers, removal of insecure ones.
> >>>> * Deprecation of ProxyTrust et al.
> >>>> * New default methods added to ServiceRegistrar to allow establising
trust with a service, prior to obtaining a service proxy, or Entry's (works with both maven
codebase provisioning and traditional codebase downloads).
> >>>> * Input validation for untrusted serial data.
> >>>> * META-INF/permissions.perm files list permissions required by codebase,
accessible from ClassLoader using mixin interface.
> >>>>
> >>>> I recall you had a need for a SocketFactory in LookupLocator, at that
time LookupLocator only used discovery v1, which was deprecated, I'd like to include a way
to enable SocketFactory support in discovery V2.  Note that LookupLocator isn't serialized
during discovery.
> >>>>
> >>>> While the River codebase was a little difficult to understand at first,
I find it's quite easy to work with now.
> >>>>
> >>>> While I'm a critic of Rivers flaws, addressing th em is straight forward,
the greatest challenge is convincing everyone that flaws exist, or that they need addressing,
there's still plenty of good stuff left worth saving.
> >>>>
> >>>> Regards,
> >>>>
> >>>> Peter.
> >>>>
> >>>>
> >>>> Sent from my Samsung device.
> >>>>
> >>>>     Include original message
> >>>> ---- Original message ----
> >>>> From: Peter<jini@zeus.net.au>
> >>>> Sent: 01/07/2016 04:35:16 pm
> >>>> To: dev@river.apache.org<dev@river.apache.org>
> >>>> Subject: Re: Lotj - languages other than java
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> Thanks Sim,
> >>>>
> >>>> These are all good questions we need to consider.
> >>>>
> >>>> I like the model of micro services where each service is responsible
for implementing its own back end persistence and state.  Do you consider a microservice to
be web based?
> >>>>
> >>>> I have an implementation of discovery using multicast ipv6.  However
for firewalls with limited open ports such as https over ipv6, we have JERI https endpoints,
but no discovery, DNS-SD is a good discovery alternative waiting to be implemented.
> >>>>
> >>>> For my own environment I will be adopting ipv6 , the global address
space and autoconfiguration solve many problems that users experience with ipv4 today.
> >>>>
> >>>> I admit the locked down api caused me frustration, but I think it's
clear now that we need a process for managing api evolution.
> >>>>
> >>>> Complexity - The proxy preparation api tries to determine trust after
downloading untrusted code and deserialization of unverified data.  As gadget attacks demonstrate,
too little too late at great complexity.  This was an attempt to bolt security onto the existing
lookup service.
> >>>>
> >>>> JERI is good, method constraints are good, proxy trust is obsolete.
 River's current ssl and https JERI endpoints need to be brought up to date as they're no
longer secure.  I've already done this work externally, it can be donated when appropriate
for the project.
> >>>>
> >>>> If we address security issues, we can provide a secure alternative to
RMI  Oracle has chosen 'whack a mole' security for Serialization, rather than address some
fundamental design flaws with ObjectInputStream, for this reason, authentication of the source
must occur prior to accepting serial data.  Despite common belief, white listing isn't a completely
secure solution and adds conplexity as it's too fine grained.
> >>>>
> >>>> For multi language support, this would limit the type system, but then,
there's a lot that can be done with strings, primitive types and byte arrays.  This doesn't
have to limit java service types, I think language support should be something determined
during lookup, so we don't limit the type systems of more powerful languages to primitives.
> >>>>
> >>>> Looking at most Entry's used for lookup, most fields are strings and
integers.  If you look at the way lookup searches are implemented, an entry is represented
by a string name and each field is a tuple name value pair.
> >>>>
> >>>> I think a ground up redesign of the lookup service, would address language
compatibility as well as complexity and security.
> >>>>
> >>>> In other words, recognise the need for a lookup&   registration
protocol, as well as discovery, after that, the service&   client should be able to negotiate
 whatever rpc protocol they have in common and to do that, we'll also need a connection negotiation
protocol.  We could write specifications for these protocols.  This way we could allow any
language/ platform to register and provide services.  The code for lookup would not be downloaded
as Reggie is now, it would be protocol, rather than proxy based.  This would also fit well
with IoT.
> >>>>
> >>>> Meanwhile we can still support existing java based services.
> >>>>
> >>>> Thoughts?
> >>>>
> >>>> Peter.
> >>>>
> >>>> Sent from my Samsung device.
> >>>>
> >>>>     Include original message
> >>>> ---- Original message ----
> >>>> From: Simon IJskes - QCG<simon@qcg.nl>
> >>>> Sent: 30/06/2016 06:22:30 pm
> >>>> To: dev@river.apache.org
> >>>> Subject: Re: Lotj - languages other than java
> >>>>
> >>>> If you solve the 'barrier' of the service discovery, do you also want
> >>>> provide universal access to the java services in the form of microservices?
> >>>>
> >>>> It is doable to take any 'more used' service discovery solution and
> >>>> this as the river discovery. To introduce a level of abstraction with
> >>>> the same primitives as the current river discovery mechanism offers.
> >>>>
> >>>> River would then have adapted a more common discovery mechanism.
> >>>>
> >>>> Next thing that we should decide, is how far do we go into universality.
> >>>> I see univeral type systems, different serialisation plugins on the
> >>>>
> >>>> The biggest showstopper for me was the API compatibility. In order to
> >>>> make any progress we need a more agile process for modifing the API
> >>>>
> >>>> If we leave compatibility behind us, we could ask our selfs, what
> >>>> benefit are we providing for the users? What can we introduce that does
> >>>> not duplicate what is already in the market For a java developer, i
> >>>> think there is no need to convince, they can see benefits in just having
> >>>> a java API to program against. We need to think about the environment
> >>>> where java receives a lot of 'non-love', how we can create a 'whow,
> >>>> isn't all that bad, look at that easy solution' experience.
> >>>>
> >>>> I think that river lost the spot it could have, as a java only solution
> >>>> to JSON, XMLRPC, SOAP, etc libraries for java. From a helicopter view,
> >>>> what does it do? Whe provide secure RPC, with discovery and scaling.
> >>>> we make it hard to use.
> >>>>
> >>>> G. Simon
> >>>>
> >>>>
> >>>> On 30-06-16 05:37, Peter wrote:
> >>>>> Currently with River, you need java to participate.  Other languages
can provide services, but you need a jvm to participate.
> >>>>>
> >>>>> Most of discovery is language agnostic, so any language can participate
in discovery.
> >>>>>
> >>>>> The major limitation for other languages is the lookup service.
 Security issues and complexity also relate to the lookup service.
> >>>>>
> >>>>> My thoughts are that a lookup service that performs search and registration,
but provides a language independent  and secure means of contacting service providers would
be beneficial.
> >>>>>
> >>>>> Anyone interested in discussing further?
> >>>>>
> >>>>> Regards,
> >>>>>
> >>>>> Peter.
> >>>>>
> >>>>>
> >>>>> Sent from my Samsung device.
> >>>>>
> >>>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>    		 	   		
> >   		 	   		
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message