river-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peter <j...@zeus.net.au>
Subject Don't let Jini Standards become an impediment to development
Date Fri, 04 Sep 2015 08:40:12 GMT
Threats to development and collaboration (as I see it):

   1. The Jini standards are sacrosanct.
   2. River is an implementation of the Jini Standards.
   3. River has no public api other than the Jini standards.
   4. Although public API can be improved in a backward compatible
      manner, the mandatory Jini Standards should not be.
   5. The Jini standards are static; there is, at present, no process
      for standards review or replacement.

This is the message I’m receiving, is this the message we want to send 
would be River developers?

Sun Labs was isolated from the rest of Sun, to ensure developers were 
able to innovate, how can River innovate now?

Here’s an example of a problem with the Jini public api, there are many 
as such, I’d like to fix, but for the sake of harmony on river-dev, I 
don't discuss them, instead I document and work around them if I can, I 
only discuss issues I can't work around:

The following is an implementation comment from 
org/apache/river/impl/lease/AbstractLeaseMap.java
/**
* AbstractLeaseMap is intended to work around some minor design warts in 
the
* {@link Lease} interface:
*
* In the real world, when a Lease is renewed, a new Lease contract document
* is issued, however when an electronic Lease is renewed the Lease expiry
* date is changed and the record of the previous Lease is lost. Ideally the
* renew method would return a new Lease.
*
* Current Lease implementations rely on a {@link Uuid} to represents the 
lease,
* the expiry date is not included the equals or hashCode calculations. 
For this
* reason, two Lease objects, one expired and one valid, may be equal, this
* is undesirable.
*
* The Lease interface doesn't specify a contract for equals or hashCode,
* all Lease implementations are also mutable, previous implementations
* of {@link LeaseMap} used Leases as keys.
*
* AbstractLeaseMap uses only the {@link ID}, usually a {@link Uuid}
* provided by a Lease for internal map keys, if {@link ID} is not 
implemented
* then the Lease itself is used as the key.
*
* Both Lease keys and Long values are actually stored internally as values
* referred to by ID keys, allowing Lease implementations to either not 
override
* hashCode and equals object methods or allow implementations that more
* accurately model reality.

Documentation from the Map interface states:

    Note: great care must be exercised if mutable objects are used as
    map keys. The behavior of a map is not specified if the value of an
    object is changed in a manner that affects equals comparisons while
    the object is a key in the map. A special case of this prohibition
    is that it is not permissible for a map to contain itself as a key.
    While it is permissible for a map to contain itself as a value,
    extreme caution is advised: the equals and hashCode methods are no
    longer well defined on such a map.

You can't tell me this is well designed, it might be standardised, but 
it's also flawed.

The first issue is, if I make an attempt to address this issue, there 
will be strong resistance to doing so. Who remembers what happened when 
I tried to create a Startable interface for starting services? It's now 
only an implementation detail? It was a very frustrating discussion, one 
developer hasn’t returned, yes I didn’t handle it well, at the time, I 
deliberately drove that developer away out of frustration.

But the outcome has not been beneficial, there were no winners, not only 
did we lose a community member, even experienced developers are still 
exporting from constructors, new users will read our examples and 
unsafely export their services from within constructors and if they're 
using River 3.0, they're going to experience problems, how's this going 
to help adoption? River 3.0 can hammer a service with multiple threads, 
running at native socket speeds, if there's a concurrency bug, River 3.0 
will expose it. With River 3.0, all hotspots are native methods.

I think with River 3.0, we need a statement in the README file that 
says, River 3.0 does not support exporting services from object 
constructors.

It's not so much a question of is River 3.0 ready for release, but is 
the world ready for River 3.0?

Too much time is consumed debating and analysing, too little on 
development, we are suffering from standards concensus paralysis. It's 
fair to say that this has a negative effect on developer motivation. We 
spend days arguing about something that takes 20 minutes to implement, 
meanwhile development stalls and people complain of slipping release dates.

During project incubation, we had a philosophy of doers decide, so if 
you wanted to veto something you needed to implement something else that 
solved the original issue. But now we have returned to a concensus model 
and we lack the ability to make progress when we cannot achieve concensus.

It sounds like Git could help us a lot with our development 
collaboration problem.

With Git, we could have an “Official Jini Standards release branch.” For 
those of us that bump into the limitations of the Jini Standards API, we 
need another branch: “Jini standards with extensions”.

In this extensions branch, we can have innovation... If parts of this 
branch gain concensus, we then integrate these stable components into 
the Jini Standards, without requiring a namespace change that breaks 
compatibility.

So think of one branch as the concensus branch and the other is the 
doer's decide branch.

Those of us who want to further River aren't a threat to the existance 
of existing implementations. Git will enable us to maintain a shared 
codebase.

For example the Lease interface could contain a default method:

public interface Lease {

final long FOREVER = Long.MAX_VALUE;
final long ANY = -1;
final int DURATION = 1;
final int ABSOLUTE = 2;
long getExpiration();
void cancel() throws UnknownLeaseException, RemoteException;
void renew(long duration)
throws LeaseDeniedException, UnknownLeaseException, RemoteException;
void setSerialFormat(int format);
int getSerialFormat();
LeaseMap<? extends Lease, Long> createLeaseMap(long duration);
boolean canBatch(Lease lease);

default Lease renewal(long duration) throws LeaseDeniedException, 
UnknownLeaseException, RemoteException
{
renew(duration);
return this;
}

}

The next thing would be to write a contract for Lease equals and 
hashcode methods.

Q: How does this help?
A: It reduces maintenance, simplifies debugging, reduces mutable state 
and clarifies ambiguity in implementing the existing interface.

All River’s Lease implementations in the Jini standards extended 
edition, would be immutable and override the new method and return a new 
copy, the renew method would be implemented so the Lease would replace 
itself in the LeaseMap. LeaseMap implementations would replace the 
existing Lease with the renewed Lease. The implications are subtle, but 
it simplifies the implementation of Leases greatly.

For those that remain sceptical, existing third party Lease 
implementations and the strict Jini standards edition would still be 
interoperable, although less defined, this would be an example of stable 
backward compatible api evolution. If there was concensus, this change 
could be incorporated into the Jini standards. The Jini standards 
extended edition would be fertile ground for developers to innovate 
without threatening the existing user base.

An example of hampered development is my recent research into River 
service security. I found that River still depends on Serialization for 
security.

No other software today relies on Serialization for security? Do we 
really want this?

The results of my investigation indicated that the serialization 
protocol wasn’t at fault, but the input stream needed to be filtered 
much like a web server filters input. Securing serialization by 
filtering input, would allow bootstrap proxy’s to be securely obtained 
from lookup services by clients.

Unfortunately due to ObjectInputStream's api, it isn't possible to have 
pluggable input filters like a web server, but it was possible to use 
the existing protocol and remain serial form compliant.

Now this atomic serialization was an implementation fix, relevant only 
to Entry’s and bootstrap proxy’s, it's selection would have been by 
configuration and security constraints. No users had to implement it; it 
wasn’t part of the api, but it would provide them with secure bootstrap 
proxy’s. If users wanted to use this in their own services, they could 
have done so, with significant performance, evolution and security 
benefits.

The consensus on river-dev was that we should limit the amount of data 
that could be downloaded through an input stream, but serialization 
wasn’t a River project concern. The sad reality is that you can’t limit 
the amount of data through the input stream, because it breaks once you 
hit that limit. The recommendation can’t be implemented, but I figured 
the odds of acceptance were low and let it go. It was shut down before I 
had opportunity to present the code for peer review.

Why isn't the message: "Ok, lets have a look at the code, can you 
explain more?" This was how the OpenJDK project responded. Due to other 
constraints, it couldn’t be fully adopted by OpenJDK, but at least they 
listened and implemented some of the functionality.

So now we live with a legacy; we have this broken proxy trust model that 
burdens users. I would still like to fix it.

As a River developer, I’m being boxed in by Jini Specifications, I can 
only fiddle with the implementation and fix bugs. I can build software 
around it, but I’m prohibited from fixing fundamental design flaws.

I had to pause and think before sending this email, I don't want to stir 
up arguments on the list, but then I also feel the need to talk about it.

I thank you for your honesty and hope you'll respect me for mine, 
perhaps we can reach a compromise that has mutual benefits, I understand 
we can't all agree, but we need to find a way to make progress when we 
don't.

Thank you,

Peter.



Mime
View raw message