river-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dawid Loubser <da...@travellinck.com>
Subject Re: Don't let Jini Standards become an impediment to development
Date Fri, 04 Sep 2015 11:27:01 GMT
Absolutely - agree with the modularisation and git.
After that, we can stick to straight semantic versioning for
compatability - i.e. the Jini API (standard) can evolve into future,
non-backwards-compatible versions where necessary.

Dawid


On 04/09/2015 13:13, Bryan Thompson wrote:
> It seems like modularization + git would go a long way toward encouraging
> parallel development, innovation, and reducing the time between releases.
>
> I think attempting to maintain both pure backward compatible jini and
> forward evolution of river seems like a path for confusion.  How are people
> to understand exactly which artifacts to deploy when there are jini and
> river flavors, or even multiple river flavors?  I would rather develop a
> means to support that evolution that works.
>
> I am all in favor of the security and concurrency improvements.
>
> Bryan
>
> ----
> Bryan Thompson
> Chief Scientist & Founder
> SYSTAP, LLC
> 4501 Tower Road
> Greensboro, NC 27410
> bryan@systap.com
> http://blazegraph.com
> http://blog.bigdata.com <http://bigdata.com>
> http://mapgraph.io
>
> Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance
> graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints
> APIs.  Blazegraph is now available with GPU acceleration using our disruptive
> technology to accelerate data-parallel graph analytics and graph query.
>
> CONFIDENTIALITY NOTICE:  This email and its contents and attachments are
> for the sole use of the intended recipient(s) and are confidential or
> proprietary to SYSTAP. Any unauthorized review, use, disclosure,
> dissemination or copying of this email or its contents or attachments is
> prohibited. If you have received this communication in error, please notify
> the sender by reply email and permanently delete all copies of the email
> and its contents and attachments.
>
> On Fri, Sep 4, 2015 at 5:26 AM, Dawid Loubser <dawid@travellinck.com> wrote:
>
>> In my opinion, the simple fact of the matter is that the existing user
>> base of Jini is really tiny, and generally what can be considered "legacy".
>> There is precious little new Jini development occurring - and probably
>> all 10 of us worldwide are on this list :-)
>>
>> I have to agree with Peter's sentiments - Jini in it's entirety, both
>> spec and river implementation - needs to evolve organically based on a
>> changing environment.
>> I am all in favour of improving and making the specification more
>> relevant to modern development, over being tied to an old, fixed
>> specification that is not evolving.
>>
>> The fact that Jini standards are static, and there is no process in
>> place to evolve them, is actually really concerning. I think they *must*
>> evolve.
>> And given the vision, experience and efforts of people like Peter, it
>> would almost be criminal to not break free from an old standard, to miss
>> the potential of what may be.
>>
>> For practical reasons, and the sanity of all involved, I think it's a
>> good idea to release River 3.0 without stirring the standards, just to
>> have a more modern,
>> massively fixed implementation available - it's been a long time since a
>> release.
>>
>> But after that, I really think the Jini standards will, and have to,
>> evolve to a changing world, or become totally irrelevant. In the
>> open-source model, nobody is going to bother with an irrelevant
>> standard, they will just badly re-invent it.
>>
>> Every day that I see some half-baked "micro services" or "distributed
>> computing" framework written in ECMAScript, which are invariably poor,
>> un-researched attempts at what OSGi or Jini already thought about, I
>> cringe. Yet, these frameworks get much more attention. The only way to
>> get Jini back into the space, is to find a killer app (and I firmly
>> believe that, and I hate this term, "Internet of things" is, and always
>> has been, it), and to get some real dynamics going with the community.
>>
>> The last thing that process wants is to be burdened by some "standard"
>> from 1998 which is, actually, not widely used at all. I'm all for
>> evolution.
>>
>> regards,
>> Dawid
>>
>>
>> On 04/09/2015 10:40, Peter wrote:
>>> Threats to development and collaboration (as I see it):
>>>
>>>   1. The Jini standards are sacrosanct.
>>>   2. River is an implementation of the Jini Standards.
>>>   3. River has no public api other than the Jini standards.
>>>   4. Although public API can be improved in a backward compatible
>>>      manner, the mandatory Jini Standards should not be.
>>>   5. The Jini standards are static; there is, at present, no process
>>>      for standards review or replacement.
>>>
>>> This is the message I’m receiving, is this the message we want to send
>>> would be River developers?
>>>
>>> Sun Labs was isolated from the rest of Sun, to ensure developers were
>>> able to innovate, how can River innovate now?
>>>
>>> Here’s an example of a problem with the Jini public api, there are
>>> many as such, I’d like to fix, but for the sake of harmony on
>>> river-dev, I don't discuss them, instead I document and work around
>>> them if I can, I only discuss issues I can't work around:
>>>
>>> The following is an implementation comment from
>>> org/apache/river/impl/lease/AbstractLeaseMap.java
>>> /**
>>> * AbstractLeaseMap is intended to work around some minor design warts
>>> in the
>>> * {@link Lease} interface:
>>> *
>>> * In the real world, when a Lease is renewed, a new Lease contract
>>> document
>>> * is issued, however when an electronic Lease is renewed the Lease expiry
>>> * date is changed and the record of the previous Lease is lost.
>>> Ideally the
>>> * renew method would return a new Lease.
>>> *
>>> * Current Lease implementations rely on a {@link Uuid} to represents
>>> the lease,
>>> * the expiry date is not included the equals or hashCode calculations.
>>> For this
>>> * reason, two Lease objects, one expired and one valid, may be equal,
>>> this
>>> * is undesirable.
>>> *
>>> * The Lease interface doesn't specify a contract for equals or hashCode,
>>> * all Lease implementations are also mutable, previous implementations
>>> * of {@link LeaseMap} used Leases as keys.
>>> *
>>> * AbstractLeaseMap uses only the {@link ID}, usually a {@link Uuid}
>>> * provided by a Lease for internal map keys, if {@link ID} is not
>>> implemented
>>> * then the Lease itself is used as the key.
>>> *
>>> * Both Lease keys and Long values are actually stored internally as
>>> values
>>> * referred to by ID keys, allowing Lease implementations to either not
>>> override
>>> * hashCode and equals object methods or allow implementations that more
>>> * accurately model reality.
>>>
>>> Documentation from the Map interface states:
>>>
>>>    Note: great care must be exercised if mutable objects are used as
>>>    map keys. The behavior of a map is not specified if the value of an
>>>    object is changed in a manner that affects equals comparisons while
>>>    the object is a key in the map. A special case of this prohibition
>>>    is that it is not permissible for a map to contain itself as a key.
>>>    While it is permissible for a map to contain itself as a value,
>>>    extreme caution is advised: the equals and hashCode methods are no
>>>    longer well defined on such a map.
>>>
>>> You can't tell me this is well designed, it might be standardised, but
>>> it's also flawed.
>>>
>>> The first issue is, if I make an attempt to address this issue, there
>>> will be strong resistance to doing so. Who remembers what happened
>>> when I tried to create a Startable interface for starting services?
>>> It's now only an implementation detail? It was a very frustrating
>>> discussion, one developer hasn’t returned, yes I didn’t handle it
>>> well, at the time, I deliberately drove that developer away out of
>>> frustration.
>>>
>>> But the outcome has not been beneficial, there were no winners, not
>>> only did we lose a community member, even experienced developers are
>>> still exporting from constructors, new users will read our examples
>>> and unsafely export their services from within constructors and if
>>> they're using River 3.0, they're going to experience problems, how's
>>> this going to help adoption? River 3.0 can hammer a service with
>>> multiple threads, running at native socket speeds, if there's a
>>> concurrency bug, River 3.0 will expose it. With River 3.0, all
>>> hotspots are native methods.
>>>
>>> I think with River 3.0, we need a statement in the README file that
>>> says, River 3.0 does not support exporting services from object
>>> constructors.
>>>
>>> It's not so much a question of is River 3.0 ready for release, but is
>>> the world ready for River 3.0?
>>>
>>> Too much time is consumed debating and analysing, too little on
>>> development, we are suffering from standards concensus paralysis. It's
>>> fair to say that this has a negative effect on developer motivation.
>>> We spend days arguing about something that takes 20 minutes to
>>> implement, meanwhile development stalls and people complain of
>>> slipping release dates.
>>>
>>> During project incubation, we had a philosophy of doers decide, so if
>>> you wanted to veto something you needed to implement something else
>>> that solved the original issue. But now we have returned to a
>>> concensus model and we lack the ability to make progress when we
>>> cannot achieve concensus.
>>>
>>> It sounds like Git could help us a lot with our development
>>> collaboration problem.
>>>
>>> With Git, we could have an “Official Jini Standards release branch.”
>>> For those of us that bump into the limitations of the Jini Standards
>>> API, we need another branch: “Jini standards with extensions”.
>>>
>>> In this extensions branch, we can have innovation... If parts of this
>>> branch gain concensus, we then integrate these stable components into
>>> the Jini Standards, without requiring a namespace change that breaks
>>> compatibility.
>>>
>>> So think of one branch as the concensus branch and the other is the
>>> doer's decide branch.
>>>
>>> Those of us who want to further River aren't a threat to the existance
>>> of existing implementations. Git will enable us to maintain a shared
>>> codebase.
>>>
>>> For example the Lease interface could contain a default method:
>>>
>>> public interface Lease {
>>>
>>> final long FOREVER = Long.MAX_VALUE;
>>> final long ANY = -1;
>>> final int DURATION = 1;
>>> final int ABSOLUTE = 2;
>>> long getExpiration();
>>> void cancel() throws UnknownLeaseException, RemoteException;
>>> void renew(long duration)
>>> throws LeaseDeniedException, UnknownLeaseException, RemoteException;
>>> void setSerialFormat(int format);
>>> int getSerialFormat();
>>> LeaseMap<? extends Lease, Long> createLeaseMap(long duration);
>>> boolean canBatch(Lease lease);
>>>
>>> default Lease renewal(long duration) throws LeaseDeniedException,
>>> UnknownLeaseException, RemoteException
>>> {
>>> renew(duration);
>>> return this;
>>> }
>>>
>>> }
>>>
>>> The next thing would be to write a contract for Lease equals and
>>> hashcode methods.
>>>
>>> Q: How does this help?
>>> A: It reduces maintenance, simplifies debugging, reduces mutable state
>>> and clarifies ambiguity in implementing the existing interface.
>>>
>>> All River’s Lease implementations in the Jini standards extended
>>> edition, would be immutable and override the new method and return a
>>> new copy, the renew method would be implemented so the Lease would
>>> replace itself in the LeaseMap. LeaseMap implementations would replace
>>> the existing Lease with the renewed Lease. The implications are
>>> subtle, but it simplifies the implementation of Leases greatly.
>>>
>>> For those that remain sceptical, existing third party Lease
>>> implementations and the strict Jini standards edition would still be
>>> interoperable, although less defined, this would be an example of
>>> stable backward compatible api evolution. If there was concensus, this
>>> change could be incorporated into the Jini standards. The Jini
>>> standards extended edition would be fertile ground for developers to
>>> innovate without threatening the existing user base.
>>>
>>> An example of hampered development is my recent research into River
>>> service security. I found that River still depends on Serialization
>>> for security.
>>>
>>> No other software today relies on Serialization for security? Do we
>>> really want this?
>>>
>>> The results of my investigation indicated that the serialization
>>> protocol wasn’t at fault, but the input stream needed to be filtered
>>> much like a web server filters input. Securing serialization by
>>> filtering input, would allow bootstrap proxy’s to be securely obtained
>>> from lookup services by clients.
>>>
>>> Unfortunately due to ObjectInputStream's api, it isn't possible to
>>> have pluggable input filters like a web server, but it was possible to
>>> use the existing protocol and remain serial form compliant.
>>>
>>> Now this atomic serialization was an implementation fix, relevant only
>>> to Entry’s and bootstrap proxy’s, it's selection would have been by
>>> configuration and security constraints. No users had to implement it;
>>> it wasn’t part of the api, but it would provide them with secure
>>> bootstrap proxy’s. If users wanted to use this in their own services,
>>> they could have done so, with significant performance, evolution and
>>> security benefits.
>>>
>>> The consensus on river-dev was that we should limit the amount of data
>>> that could be downloaded through an input stream, but serialization
>>> wasn’t a River project concern. The sad reality is that you can’t
>>> limit the amount of data through the input stream, because it breaks
>>> once you hit that limit. The recommendation can’t be implemented, but
>>> I figured the odds of acceptance were low and let it go. It was shut
>>> down before I had opportunity to present the code for peer review.
>>>
>>> Why isn't the message: "Ok, lets have a look at the code, can you
>>> explain more?" This was how the OpenJDK project responded. Due to
>>> other constraints, it couldn’t be fully adopted by OpenJDK, but at
>>> least they listened and implemented some of the functionality.
>>>
>>> So now we live with a legacy; we have this broken proxy trust model
>>> that burdens users. I would still like to fix it.
>>>
>>> As a River developer, I’m being boxed in by Jini Specifications, I can
>>> only fiddle with the implementation and fix bugs. I can build software
>>> around it, but I’m prohibited from fixing fundamental design flaws.
>>>
>>> I had to pause and think before sending this email, I don't want to
>>> stir up arguments on the list, but then I also feel the need to talk
>>> about it.
>>>
>>> I thank you for your honesty and hope you'll respect me for mine,
>>> perhaps we can reach a compromise that has mutual benefits, I
>>> understand we can't all agree, but we need to find a way to make
>>> progress when we don't.
>>>
>>> Thank you,
>>>
>>> Peter.
>>>
>>>
>>
>>



Mime
View raw message