river-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Richard Nicholson <richard.nichol...@paremus.com>
Subject Re: Maven Build
Date Wed, 16 Nov 2016 13:15:05 GMT

I’d encourage the active River committers to review the OSGi Alliance RS / RSA specifications.
These specifications don’t deal with dynamic code download at present - but no reason why
they should in future - if there is a compelling Use Case.  The OSGi community and the OSGi
Alliance are very open to ideas.

Best Wishes

Richard 


Richard Nicholson
Chief Executive Officer
OSGi Alliance Board Member & Strategy






url:	http://www.paremus.com <http://www.paremus.com/>
blog:   http://adaptevolve.paremus.com <http://adaptevolve.paremus.com/>


Direct:	+44 (0) 122 445 1260
Mobile:	+44 (0) 797 105 1645
Indirect:	+44 (0) 207 936 9098
Fax:		+44 (0) 845 127 5999


_________________________________
Paremus Limited. Registered in England. Registration No. 4181472
Registered Office: 22-24 Broad Street, Wokingham, Berks RG40 1BA
Postal Address: 107-111 Fleet Street, London, EC4A 2AB

The information transmitted is intended only for the person(s) or entity to which it is addressed
and may contain confidential and/or privileged material. Any review, retransmission, dissemination
or other use of, or taking of any action in reliance upon, this information by persons or
entities other than the intended recipient is prohibited. If you received this in error, please
contact the sender and delete the material from any computer.

> On 16 Nov 2016, at 12:43, Michał Kłeczek (XPro Sp. z o. o.) <michal.kleczek@xpro.biz>
wrote
> :
> 
> 1. I am aware of both Remote Services and Remote Services Admin specifications. I would
be happy to see how they deal with dynamic code downloading but unfortunately they simply
don't:
> "This specification does not mandate the technique used to proxy an Endpoint as a service
in the OSGi
> framework." - taken from "OSGi Compendium Release 6 Specification"
> See point 3. below for some inherent issues with OSGi that make it difficult to implement
marshalling/unmarshalling as required by River.
> 
> 2. It has been a couple of years now since I last checked how Paremus implemented Jini
components. At that time the whole dynamic code downloading has been based on PreferredClassProvider
and "java.rmi.codebase" property which would suggest it did not really solve issues with code
downloading in Jini at all. I would be happy to be proven wrong.
> 
> 3. My comment about OSGi being "non-deterministic" in resolving dependencies means that
the same bundle installed in two different environments is going to be linked with different
dependent bundles. It basically means the same object graph valid in one VM instance cannot
be deserialized in another VM instance unless:
> a) all bundles have their dependencies specified in such a way that exact same dependency
graph is going to be present everywhere (meaning no Package-Imports, no version ranges etc)
> b) all JVMs have the same set of bundles installed (ie - there is no dynamic code downloading
at all - the code is preloaded)
> c) classes of serialized objects are not loaded by OSGi framework class loaders but by
other non-related class loading framework (which is what original old OSGi-Jini spec did)
> d) classes of all objects in the object graph are from the same bundle (special case
of a) )
> 
> In general all efforts I am aware of regarding "remoting" in Java didn't really even
try to solve issues of dynamically downloaded code. They were based on all parties having
the same code installed and/or having a central authority providing a shared consistent view
of all available software.
> I am not saying it is wrong. I am only saying that IMHO it is not something River should
do - it is simply a solved problem and there is not point of re-doing it (JERI is cool but
it is simply yet-another-RPC-stack)
> 
> Regarding JBoss Modules:
> I am not really advocating this particular library - it is just that it is the only (non-OSGi
- see above :) ) implementation of non-hierarchical class loading that I am aware of which
is of good quality and actively maintained. I wouldn't want to reimplement it by myself. Thought
about ClassWorlds but it doesn't seem to be too active and had some performance problems in
the past. JBoss Modules is Apache licensed so it can be used by River.
> 
> Thanks,
> Michal
> 
> 
> On Wednesday, 16 November 2016, Richard Nicholson <richard.nicholson@paremus.com <mailto:richard.nicholson@paremus.com>>
wrote:
> Agree with Niclas. I don’t understand the resolution comment.
> 
> Not only is OSGi <> Jini integration feasible but it is a historical fact. Paremus
did this - and a pretty good job we did - in … something like 2006 :-/
> 
> If the River community decided that there is interest in OSGi - then I’d suggest reading
the Remote Service and Remote Service Admin specifications and thinking about how Jini concepts
might enhance that world view. There are well over 10 million OSGi enabled IoT gateways out
there!
> 
> Sorry - I see no compelling technical or commercial arguments for the JBoss Module route.
> 
> 
> > On 16 Nov 2016, at 08:11, Niclas Hedhman <niclas@hedhman.org <javascript:;>>
wrote:
> >
> > I am curious, what do you mean by "non-deterministic dependency resolution"
> > ?  You can make it as predictable as you wish with attributes and
> > directives.
> >
> > Cheers
> > Niclas
> >
> > On Wed, Nov 16, 2016 at 4:07 PM, Michał Kłeczek <michal.kleczek@xpro.biz <javascript:;>>
> > wrote:
> >
> >> While non-hierarchical class loading is crucial, OSGI with its
> >> non-deterministic dependency resolution is very difficult ( if not
> >> impossible ) to target.
> >> I'm working on JBoss Module based class loading for River which I'm going
> >> to propose as contribution soon.
> >>
> >> Thanks,
> >> Michal
> >>
> >> On Wednesday, 16 November 2016, Dawid Loubser <dawid@travellinck.com <javascript:;>>
> >> wrote:
> >>
> >>> +1 for OSGi providing the best solution to the class resolution problem,
> >>> though I think some work will have to be done around trust, as you say.
> >>>
> >>>
> >>> On 16/11/2016 02:23, Peter wrote:
> >>>>
> >>>> The conventional alternatives will remain;  the existing ClassLoader
> >>> isolation and the complexities surrounding multiple copies of the same or
> >>> different versions of the same classes interacting within the same jvm.
> >>> Maven will present a new alternative of maximum sharing, where different
> >>> service principals will share the same identity.
> >>>>
> >>>> Clearly, the simplest solution is to avoid code download and only use
> >>> reflection proxy's
> >>>>
> >>>> An inter process call isn't remote, but there is a question of how a
> >>> reflection proxy should behave when a subprocess is terminated.
> >>>>
> >>>> UndeclaredThrowableException seems appropriate.
> >>>>
> >>>> It would plug in via the existing ClassLoading RMIClassLoader provider
> >>> mechanism, it would be a client concern, transparent to the service or
> >>> server.
> >>>>
> >>>> The existing behaviour would remain default.
> >>>>
> >>>> So there can be multiple class resolution options:
> >>>>
> >>>> 1. Existing PrefferedClassProvider.
> >>>> 2. Maven class resolution, where maximum class sharing exists.  This
> >> may
> >>> be preferable in situations where there is one domain of trust, eg within
> >>> one corporation or company.  Max performance.
> >>>> 3. Process Isolation.  Interoperation between trusted entities, where
> >>> code version incompatibilities may exist, because of separate development
> >>> teams and administrators.  Each domain of trust has it's own process
> >>> domain.  Max compatibility, but slower.
> >>>> 4. OSGi.
> >>>>
> >>>> There may be occassions where simpler (because developers don't need
to
> >>> understand ClassLoaders), slow, compatible and reliable wins over fast
> >> and
> >>> complex or broken.
> >>>>
> >>>> A subprocess may host numerous proxy's and codebases from one principal
> >>> trust domain (even a later version of River could be provisioned using
> >>> Maven).  A subprocess would exist for each trust domain. So if there are
> >>> two companies, code from each remains isolated and communicates only
> >> using
> >>> common api.  No unintended code versioning conflicts.
> >>>>
> >>>> This choice would not prevent or exclude other methods of
> >> communication,
> >>> the service, even if isolated within it's own process will still
> >>> communicate remotely over the network using JERI, JSON etc.  This is
> >>> orthogonal to and independant of remote communication protocols.
> >>>>
> >>>> OSGi would of course be an alternative option, if one wished to execute
> >>> incompatible versions of libraries etc within one process, but different
> >>> trust domains will have a shared identity, again this may not matter
> >>> depending on the use case.
> >>>>
> >>>> Cheers,
> >>>>
> >>>> Peter.
> >>>>
> >>>> ESent from my Samsung device.
> >>>>
> >>>>  Include original message
> >>>> ---- Original message ----
> >>>> From: "Michał Kłeczek (XPro Sp. z o. o.)" <michal.kleczek@xpro.biz
<javascript:;>
> >>> <javascript:;> <javascript:;>>
> >>>> Sent: 15/11/2016 10:30:29 pm
> >>>> To: dev@river.apache.org <javascript:;> <javascript:;> <javascript:;>
> >>>> Subject: Re: Maven Build
> >>>>
> >>>> While I also thought about out-of-process based mechanism for execution
> >>> of dynamically downloaded code, I came to the conclusion that in the
> >>> context of River/Java in-process mechanism is something that MUST be done
> >>> right. All other things can (and should) be built on that.
> >>>>
> >>>> I think that the proposal to implement "remote calls on smart proxy
> >>> interfaces that aren't remote" is somewhat a misnomer. The call is either
> >>> remote or local - you cannot have both at the same time. AFAIK Jini
> >>> community always believed there is no possibility to have local/remote
> >>> transparency. That is why there exists java.rmi.Remote marker interface
> >> in
> >>> the first place.
> >>>>
> >>>> There is also the question about the level of isolation you want to
> >>> achieve. Simple "out-of-process" is not enough, chroot is not enough,
> >>> CGROUPS/containers/jails/zones might be not enough, virtual machines
> >> might
> >>> be not enough :) - going the route you propose opens up the whole world
> >> of
> >>> new questions to answer. At the same time you loose the most important
> >>> advantages of in-process execution:
> >>>> - simplicity of communication between components (basic function call,
> >>> no need to do anything complicated to implement callbacks etc.)
> >>>> - performance
> >>>>
> >>>> In the end you either standardize on the well known set of
> >> communication
> >>> protocols (such as JERI) OR you say "end of protocols" by allowing
> >>> execution of dynamically downloaded code in-process.
> >>>> If River is going to choose the first route - IMHO it is going to fail
> >>> since it does not propose anything competitive comparing to current
> >>> mainstream HTTP(S)/REST/JSON stack.
> >>>>
> >>>> Thanks,
> >>>> Michal
> >>>> Peter November 15, 2016 at 8:28 AM
> >>>> I've been thinking about process isolation (instead of using
> >>> ClassLoader's for isolation).  Typically, smart proxy's are isolated in
> >>> their own ClassLoader, with their own copies of classes, however with
> >>> Maven, a lot more class sharing occurs.  Since River uses codebase
> >>> annotations for identity, using maven codebase annotations will result in
> >>> proxy's from different services sharing identity.
> >>>>
> >>>> A better way to provide for different identities coexisting on the same
> >>> node, would be to use subprocess jvm's for each Service's server
> >> principal
> >>> identity, to keep classes from different services in different processes.
> >>>>
> >>>> This way, each principal would have their own process & Maven namespace
> >>> for their proxy's.
> >>>>
> >>>> Presently JERI only exports interfaces in reflection proxy's that
> >>> implement Remote, so I'd need an endpoint that can export all
> >> interfaces,
> >>> accross a local interprocess connection to allow remote calls on smart
> >>> proxy interfaces that aren't remote.
> >>>>
> >>>> This also means that memory resource consumption of smart proxy's can
> >> be
> >>> controlled by the client and a smart proxy's process killed without
> >> killing
> >>> the client jvm.
> >>>>
> >>>> Cheers,
> >>>>
> >>>> Peter.
> >>>>
> >>>>
> >>>>
> >>>> Dawid Loubser November 15, 2016 at 8:50 AM
> >>>> As a very heavy Maven user, I wanted to say that this is great news.
> >>>> This is encouraging indeed!
> >>>>
> >>>> Dawid
> >>>>
> >>>>
> >>>> Peter November 15, 2016 at 4:08 AM
> >>>> Some other news that might  encourage participation, I've been working
> >>> on Dennis Reedy's script to modularise the codebase, I haven't run the
> >> test
> >>> suites against it and it isn't generating stubs yet, and I'll need to
> >>> modify the platform modules for the IoT effort after the conversion is
> >>> complete.
> >>>>
> >>>> Here's the output of the River maven build:
> >>>>
> >>>> Reactor Summary:
> >>>>
> >>>> River-Internet Project ............................ SUCCESS [0.689s]
> >>>> Module :: River Policy ............................ SUCCESS [8.395s]
> >>>> Module :: River Resources ......................... SUCCESS [0.607s]
> >>>> Module :: River Platform .......................... SUCCESS  [23.521s]
> >>>> Module :: River Service DL Library ................ SUCCESS [8.999s]
> >>>> Module :: River Service Library ................... SUCCESS [8.014s]
> >>>> Module :: River Service Starter ................... SUCCESS [3.930s]
> >>>> Module :: River SharedGroup Destroy ............... SUCCESS [3.018s]
> >>>> Module :: Outrigger .............................. SUCCESS [0.056s]
> >>>> Module :: Outrigger Service Download classes ...... SUCCESS [2.416s]
> >>>> Module :: Outrigger Service Implementation ........ SUCCESS [4.118s]
> >>>> Module :: Outrigger Snaplogstore ................. SUCCESS [3.273s]
> >>>> Module :: Lookup Service ......................... SUCCESS [0.048s]
> >>>> Module :: Reggie Service Download classes ........ SUCCESS [3.966s]
> >>>> Module :: Reggie Service Implementation .......... SUCCESS [3.621s]
> >>>> Module :: Mahalo ................................. SUCCESS [0.436s]
> >>>> Module :: Mahalo Service Download classes ......... SUCCESS [2.059s]
> >>>> Module :: Mahalo Service Implementation ........... SUCCESS [4.175s]
> >>>> Module :: Mercury the Event Mailbox ............... SUCCESS [0.497s]
> >>>> Module :: Mercury Service Download classes ........ SUCCESS [3622s]
> >>>> Module :: Mercury Service Implementation .......... SUCCESS [3.562s]
> >>>> Module :: Norm .................................... SUCCESS [0.013s]
> >>>> Module :: Norm Service Download classes ........... SUCCESS [2.867s]
> >>>> Module :: Norm Service Implementation ............. SUCCESS [6.390s]
> >>>> Module :: Group ................................... SUCCESS [0.025s]
> >>>> Module :: Mahalo Service Download classes ......... SUCCESS [2.877s]
> >>>> Module :: Group Service Implementation ............ SUCCESS [2.037s]
> >>>> Module :: Tools ................................... SUCCESS [0.485s]
> >>>> Module :: Check ConfigurationFile ................. SUCCESS [2.720s]
> >>>> Module :: Check serialversionUid .................. SUCCESS [2.129s]
> >>>> Module :: ClassDep ................................ SUCCESS [4.157s]
> >>>> Module :: Class Server ............................. SUCCESS [3.353s]
> >>>> Module :: Compute message digest .................. SUCCESS [1.734s]
> >>>> Module :: Compute httpmd codebase ................. SUCCESS [2.102s]
> >>>> Module :: Environment Check ...................... SUCCESS [2.837s]
> >>>> Module :: Jar wrapper ............................ SUCCESS [2.179s]
> >>>> Module :: Preferred classes list generator ........ SUCCESS [2.495s]
> >>>> Module :: Phoenix Activation ..................... SUCCESS [0.029s]
> >>>> Module :: Phoenix Download ....................... SUCCESS [2.685s]
> >>>> Module :: Phoenix ................................ SUCCESS [4.095s]
> >>>> Module :: Phoenix Group ........................... SUCCESS [2.445s]
> >>>> Module :: Phoenix Init ............................ SUCCESS [1.740s]
> >>>> River Distribution ................................ SUCCESS [10.523s]
> >>>> ------------------------------------------------------------
> >> ------------
> >>>> BUILD SUCCESS
> >>>> ------------------------------------------------------------
> >> ------------
> >>>> Total time: 2:29.804s
> >>>> Finished at: Mon Nov 14 22:22:31 EST 2016
> >>>> Final Memory: 145M/247M
> >>>> ------------------------------------------------------------
> >> ------------
> >>>>
> >>>>
> >>>>
> >>>>
> >>>
> >>>
> >>
> >
> >
> >
> > --
> > Niclas Hedhman, Software Developer
> > http://zest.apache.org <http://zest.apache.org/> - New Energy for Java
> 


Mime
View raw message