river-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michał Kłeczek (XPro Sp. z o. o.)" <michal.klec...@xpro.biz>
Subject Re: Maven Build
Date Wed, 16 Nov 2016 12:43:42 GMT
1. I am aware of both Remote Services and Remote Services Admin 
specifications. I would be happy to see how they deal with dynamic code 
downloading but unfortunately they simply don't:
"This specification does not mandate the technique used to proxy an 
Endpoint as a service in the OSGi
framework." - taken from "OSGi Compendium Release 6 Specification"
See point 3. below for some inherent issues with OSGi that make it 
difficult to implement marshalling/unmarshalling as required by River.

2. It has been a couple of years now since I last checked how Paremus 
implemented Jini components. At that time the whole dynamic code 
downloading has been based on PreferredClassProvider and 
"java.rmi.codebase" property which would suggest it did not really solve 
issues with code downloading in Jini at all. I would be happy to be 
proven wrong.

3. My comment about OSGi being "non-deterministic" in resolving 
dependencies means that the same bundle installed in two different 
environments is going to be linked with different dependent bundles. It 
basically means the same object graph valid in one VM instance cannot be 
deserialized in another VM instance unless:
a) all bundles have their dependencies specified in such a way that 
exact same dependency graph is going to be present everywhere (meaning 
no Package-Imports, no version ranges etc)
b) all JVMs have the same set of bundles installed (ie - there is no 
dynamic code downloading at all - the code is preloaded)
c) classes of serialized objects are not loaded by OSGi framework class 
loaders but by other non-related class loading framework (which is what 
original old OSGi-Jini spec did)
d) classes of all objects in the object graph are from the same bundle 
(special case of a) )

In general all efforts I am aware of regarding "remoting" in Java didn't 
really even try to solve issues of dynamically downloaded code. They 
were based on all parties having the same code installed and/or having a 
central authority providing a shared consistent view of all available 
software.
I am not saying it is wrong. I am only saying that IMHO it is not 
something River should do - it is simply a solved problem and there is 
not point of re-doing it (JERI is cool but it is simply 
yet-another-RPC-stack)

Regarding JBoss Modules:
I am not really advocating this particular library - it is just that it 
is the only (non-OSGi - see above :) ) implementation of 
non-hierarchical class loading that I am aware of which is of good 
quality and actively maintained. I wouldn't want to reimplement it by 
myself. Thought about ClassWorlds but it doesn't seem to be too active 
and had some performance problems in the past. JBoss Modules is Apache 
licensed so it can be used by River.

Thanks,
Michal


On Wednesday, 16 November 2016, Richard Nicholson 
<richard.nicholson@paremus.com <mailto:richard.nicholson@paremus.com>> 
wrote:

    Agree with Niclas. I don’t understand the resolution comment.

    Not only is OSGi <> Jini integration feasible but it is a historical
    fact. Paremus did this - and a pretty good job we did - in …
    something like 2006 :-/

    If the River community decided that there is interest in OSGi - then
    I’d suggest reading the Remote Service and Remote Service Admin
    specifications and thinking about how Jini concepts might enhance
    that world view. There are well over 10 million OSGi enabled IoT
    gateways out there!

    Sorry - I see no compelling technical or commercial arguments for
    the JBoss Module route.


     > On 16 Nov 2016, at 08:11, Niclas Hedhman <niclas@hedhman.org
    <javascript:;>> wrote:
     >
     > I am curious, what do you mean by "non-deterministic dependency
    resolution"
     > ?  You can make it as predictable as you wish with attributes and
     > directives.
     >
     > Cheers
     > Niclas
     >
     > On Wed, Nov 16, 2016 at 4:07 PM, Michał Kłeczek
    <michal.kleczek@xpro.biz <javascript:;>>
     > wrote:
     >
     >> While non-hierarchical class loading is crucial, OSGI with its
     >> non-deterministic dependency resolution is very difficult ( if not
     >> impossible ) to target.
     >> I'm working on JBoss Module based class loading for River which
    I'm going
     >> to propose as contribution soon.
     >>
     >> Thanks,
     >> Michal
     >>
     >> On Wednesday, 16 November 2016, Dawid Loubser
    <dawid@travellinck.com <javascript:;>>
     >> wrote:
     >>
     >>> +1 for OSGi providing the best solution to the class resolution
    problem,
     >>> though I think some work will have to be done around trust, as
    you say.
     >>>
     >>>
     >>> On 16/11/2016 02:23, Peter wrote:
     >>>>
     >>>> The conventional alternatives will remain;  the existing
    ClassLoader
     >>> isolation and the complexities surrounding multiple copies of
    the same or
     >>> different versions of the same classes interacting within the
    same jvm.
     >>> Maven will present a new alternative of maximum sharing, where
    different
     >>> service principals will share the same identity.
     >>>>
     >>>> Clearly, the simplest solution is to avoid code download and
    only use
     >>> reflection proxy's
     >>>>
     >>>> An inter process call isn't remote, but there is a question of
    how a
     >>> reflection proxy should behave when a subprocess is terminated.
     >>>>
     >>>> UndeclaredThrowableException seems appropriate.
     >>>>
     >>>> It would plug in via the existing ClassLoading RMIClassLoader
    provider
     >>> mechanism, it would be a client concern, transparent to the
    service or
     >>> server.
     >>>>
     >>>> The existing behaviour would remain default.
     >>>>
     >>>> So there can be multiple class resolution options:
     >>>>
     >>>> 1. Existing PrefferedClassProvider.
     >>>> 2. Maven class resolution, where maximum class sharing
    exists.  This
     >> may
     >>> be preferable in situations where there is one domain of trust,
    eg within
     >>> one corporation or company.  Max performance.
     >>>> 3. Process Isolation.  Interoperation between trusted
    entities, where
     >>> code version incompatibilities may exist, because of separate
    development
     >>> teams and administrators.  Each domain of trust has it's own
    process
     >>> domain.  Max compatibility, but slower.
     >>>> 4. OSGi.
     >>>>
     >>>> There may be occassions where simpler (because developers
    don't need to
     >>> understand ClassLoaders), slow, compatible and reliable wins
    over fast
     >> and
     >>> complex or broken.
     >>>>
     >>>> A subprocess may host numerous proxy's and codebases from one
    principal
     >>> trust domain (even a later version of River could be
    provisioned using
     >>> Maven).  A subprocess would exist for each trust domain. So if
    there are
     >>> two companies, code from each remains isolated and communicates
    only
     >> using
     >>> common api.  No unintended code versioning conflicts.
     >>>>
     >>>> This choice would not prevent or exclude other methods of
     >> communication,
     >>> the service, even if isolated within it's own process will still
     >>> communicate remotely over the network using JERI, JSON etc. 
    This is
     >>> orthogonal to and independant of remote communication protocols.
     >>>>
     >>>> OSGi would of course be an alternative option, if one wished
    to execute
     >>> incompatible versions of libraries etc within one process, but
    different
     >>> trust domains will have a shared identity, again this may not
    matter
     >>> depending on the use case.
     >>>>
     >>>> Cheers,
     >>>>
     >>>> Peter.
     >>>>
     >>>> ESent from my Samsung device.
     >>>>
     >>>>  Include original message
     >>>> ---- Original message ----
     >>>> From: "Michał Kłeczek (XPro Sp. z o. o.)"
    <michal.kleczek@xpro.biz <javascript:;>
     >>> <javascript:;>>
     >>>> Sent: 15/11/2016 10:30:29 pm
     >>>> To: dev@river.apache.org <javascript:;> <javascript:;>
     >>>> Subject: Re: Maven Build
     >>>>
     >>>> While I also thought about out-of-process based mechanism for
    execution
     >>> of dynamically downloaded code, I came to the conclusion that
    in the
     >>> context of River/Java in-process mechanism is something that
    MUST be done
     >>> right. All other things can (and should) be built on that.
     >>>>
     >>>> I think that the proposal to implement "remote calls on smart
    proxy
     >>> interfaces that aren't remote" is somewhat a misnomer. The call
    is either
     >>> remote or local - you cannot have both at the same time. AFAIK Jini
     >>> community always believed there is no possibility to have
    local/remote
     >>> transparency. That is why there exists java.rmi.Remote marker
    interface
     >> in
     >>> the first place.
     >>>>
     >>>> There is also the question about the level of isolation you
    want to
     >>> achieve. Simple "out-of-process" is not enough, chroot is not
    enough,
     >>> CGROUPS/containers/jails/zones might be not enough, virtual
    machines
     >> might
     >>> be not enough :) - going the route you propose opens up the
    whole world
     >> of
     >>> new questions to answer. At the same time you loose the most
    important
     >>> advantages of in-process execution:
     >>>> - simplicity of communication between components (basic
    function call,
     >>> no need to do anything complicated to implement callbacks etc.)
     >>>> - performance
     >>>>
     >>>> In the end you either standardize on the well known set of
     >> communication
     >>> protocols (such as JERI) OR you say "end of protocols" by allowing
     >>> execution of dynamically downloaded code in-process.
     >>>> If River is going to choose the first route - IMHO it is going
    to fail
     >>> since it does not propose anything competitive comparing to current
     >>> mainstream HTTP(S)/REST/JSON stack.
     >>>>
     >>>> Thanks,
     >>>> Michal
     >>>> Peter November 15, 2016 at 8:28 AM
     >>>> I've been thinking about process isolation (instead of using
     >>> ClassLoader's for isolation).  Typically, smart proxy's are
    isolated in
     >>> their own ClassLoader, with their own copies of classes,
    however with
     >>> Maven, a lot more class sharing occurs.  Since River uses codebase
     >>> annotations for identity, using maven codebase annotations will
    result in
     >>> proxy's from different services sharing identity.
     >>>>
     >>>> A better way to provide for different identities coexisting on
    the same
     >>> node, would be to use subprocess jvm's for each Service's server
     >> principal
     >>> identity, to keep classes from different services in different
    processes.
     >>>>
     >>>> This way, each principal would have their own process & Maven
    namespace
     >>> for their proxy's.
     >>>>
     >>>> Presently JERI only exports interfaces in reflection proxy's that
     >>> implement Remote, so I'd need an endpoint that can export all
     >> interfaces,
     >>> accross a local interprocess connection to allow remote calls
    on smart
     >>> proxy interfaces that aren't remote.
     >>>>
     >>>> This also means that memory resource consumption of smart
    proxy's can
     >> be
     >>> controlled by the client and a smart proxy's process killed without
     >> killing
     >>> the client jvm.
     >>>>
     >>>> Cheers,
     >>>>
     >>>> Peter.
     >>>>
     >>>>
     >>>>
     >>>> Dawid Loubser November 15, 2016 at 8:50 AM
     >>>> As a very heavy Maven user, I wanted to say that this is great
    news.
     >>>> This is encouraging indeed!
     >>>>
     >>>> Dawid
     >>>>
     >>>>
     >>>> Peter November 15, 2016 at 4:08 AM
     >>>> Some other news that might  encourage participation, I've been
    working
     >>> on Dennis Reedy's script to modularise the codebase, I haven't
    run the
     >> test
     >>> suites against it and it isn't generating stubs yet, and I'll
    need to
     >>> modify the platform modules for the IoT effort after the
    conversion is
     >>> complete.
     >>>>
     >>>> Here's the output of the River maven build:
     >>>>
     >>>> Reactor Summary:
     >>>>
     >>>> River-Internet Project ............................ SUCCESS
    [0.689s]
     >>>> Module :: River Policy ............................ SUCCESS
    [8.395s]
     >>>> Module :: River Resources ......................... SUCCESS
    [0.607s]
     >>>> Module :: River Platform .......................... SUCCESS 
    [23.521s]
     >>>> Module :: River Service DL Library ................ SUCCESS
    [8.999s]
     >>>> Module :: River Service Library ................... SUCCESS
    [8.014s]
     >>>> Module :: River Service Starter ................... SUCCESS
    [3.930s]
     >>>> Module :: River SharedGroup Destroy ............... SUCCESS
    [3.018s]
     >>>> Module :: Outrigger .............................. SUCCESS
    [0.056s]
     >>>> Module :: Outrigger Service Download classes ...... SUCCESS
    [2.416s]
     >>>> Module :: Outrigger Service Implementation ........ SUCCESS
    [4.118s]
     >>>> Module :: Outrigger Snaplogstore ................. SUCCESS
    [3.273s]
     >>>> Module :: Lookup Service ......................... SUCCESS
    [0.048s]
     >>>> Module :: Reggie Service Download classes ........ SUCCESS
    [3.966s]
     >>>> Module :: Reggie Service Implementation .......... SUCCESS
    [3.621s]
     >>>> Module :: Mahalo ................................. SUCCESS
    [0.436s]
     >>>> Module :: Mahalo Service Download classes ......... SUCCESS
    [2.059s]
     >>>> Module :: Mahalo Service Implementation ........... SUCCESS
    [4.175s]
     >>>> Module :: Mercury the Event Mailbox ............... SUCCESS
    [0.497s]
     >>>> Module :: Mercury Service Download classes ........ SUCCESS
    [3622s]
     >>>> Module :: Mercury Service Implementation .......... SUCCESS
    [3.562s]
     >>>> Module :: Norm .................................... SUCCESS
    [0.013s]
     >>>> Module :: Norm Service Download classes ........... SUCCESS
    [2.867s]
     >>>> Module :: Norm Service Implementation ............. SUCCESS
    [6.390s]
     >>>> Module :: Group ................................... SUCCESS
    [0.025s]
     >>>> Module :: Mahalo Service Download classes ......... SUCCESS
    [2.877s]
     >>>> Module :: Group Service Implementation ............ SUCCESS
    [2.037s]
     >>>> Module :: Tools ................................... SUCCESS
    [0.485s]
     >>>> Module :: Check ConfigurationFile ................. SUCCESS
    [2.720s]
     >>>> Module :: Check serialversionUid .................. SUCCESS
    [2.129s]
     >>>> Module :: ClassDep ................................ SUCCESS
    [4.157s]
     >>>> Module :: Class Server ............................. SUCCESS
    [3.353s]
     >>>> Module :: Compute message digest .................. SUCCESS
    [1.734s]
     >>>> Module :: Compute httpmd codebase ................. SUCCESS
    [2.102s]
     >>>> Module :: Environment Check ...................... SUCCESS
    [2.837s]
     >>>> Module :: Jar wrapper ............................ SUCCESS
    [2.179s]
     >>>> Module :: Preferred classes list generator ........ SUCCESS
    [2.495s]
     >>>> Module :: Phoenix Activation ..................... SUCCESS
    [0.029s]
     >>>> Module :: Phoenix Download ....................... SUCCESS
    [2.685s]
     >>>> Module :: Phoenix ................................ SUCCESS
    [4.095s]
     >>>> Module :: Phoenix Group ........................... SUCCESS
    [2.445s]
     >>>> Module :: Phoenix Init ............................ SUCCESS
    [1.740s]
     >>>> River Distribution ................................ SUCCESS
    [10.523s]
     >>>> ------------------------------------------------------------
     >> ------------
     >>>> BUILD SUCCESS
     >>>> ------------------------------------------------------------
     >> ------------
     >>>> Total time: 2:29.804s
     >>>> Finished at: Mon Nov 14 22:22:31 EST 2016
     >>>> Final Memory: 145M/247M
     >>>> ------------------------------------------------------------
     >> ------------
     >>>>
     >>>>
     >>>>
     >>>>
     >>>
     >>>
     >>
     >
     >
     >
     > --
     > Niclas Hedhman, Software Developer
     > http://zest.apache.org - New Energy for Java


Mime
  • Unnamed multipart/mixed (inline, None, 0 bytes)
View raw message