qpid-proton mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Fraser Adams <fraser.ad...@blueyonder.co.uk>
Subject Re: improving cross language maintainability
Date Fri, 20 Dec 2013 16:43:00 GMT
I've been following this thread with interest and I guess that Rob's comment

"

However I think that would require us to be
organised differently with a recognition that the Engine and Messenger are
conceptually separate (with Messenger depending on the Engine API, and
having no more authority over defining/testing the engine API than any
other client
"

Struck some resonance.

I think perhaps the layering is OK if you're familiar with the code base and perhaps it's
more about "packaging" than layering, but FWIW coming into proton quite "cold" it seems like
messenger and engine are essentially "combined". Now I know that's not the case now from the
responses to my "how does it all hang together" question of a few weeks ago, but in terms
of packaging that's still kind of the case.

So what I mean by that is that qpid::messaging and proton messenger are as far as has been
discussed with me "peer" APIs - both high level that can achieve similar things albeit with
different semantics, whereas engine is a lower level API.

Why is it in that case that the proton library is built that contains both messenger and engine?
OK, so it's convenient, but as far as I'm aware neither qpid::messaging nor Ted's dispatch
router actually use messenger at all, they both use engine? That would suggest to me that
engine and messenger probably ought to be exposed as separate libraries? (and presumably there's
a similar position in Java where the JMS implementation is engine not messenger based - though
I've not looked at the jar packaging).

I guess (perhaps related) it just might be better if messenger was in a separate source tree,
which I'd agree might be a faff, but would clarify the hierarchical rather than peer relationship
between messenger and engine.

So I guess that my take is that Rafael is perfectly accurate when he says that "I believe
they are currently layered precisely the way you
describe" that certainly seems to be the case at the logical level, but at the "physical"
level it's an awful lot less clear of the layering - certainly from the perspective of a novice
trying to navigate the code base.

I'd have to agree also with Rob about the horizontal scaling aspects of lack of comments and
documentation, particularly with respect to engine. I'm currently trying to get a JavaScript_messenger_
 working because that seems tractable given the examples and documentation around messenger
(though as you know even there I've had a fair few complications) my personal preference/familiarity
is however around the JMS/qpid::messaging API and I'd ultimately quite like to have a JavaScript
"binding" for that, but at this stage I wouldn't know where to begin and I'd probably end
up "reverse engineering" the work Gordon has done with qpid::messaging and/or Rob and co.
are doing with the new JMS implementation.

I'd be interested to hear from Gordon and Ted about how they went about building capability
on top of the engine API.

I hope I'm not coming across as over-critical - email is a bit rubbish at conveying emotion
:-) I guess that I just want to add my 2p from the perspective of someone coming in new with
no previous exposure to the code base, which is hopefully a fairly objective position.
Best regards,
Frase


On 20/12/13 15:57, Rafael Schloming wrote:
> On Fri, Dec 20, 2013 at 5:46 AM, Rob Godfrey <rob.j.godfrey@gmail.com>wrote:
>
>> Since I have pretty much given up on trying to work on proton, I'm not sure
>> that my opinion counts for much anymore... however I suppose I'm a little
>> curious how the JNI binding differs in "extra work" vs. PHP or Ruby or
>> such. Similarly why the API distinction is "hasty" when again it should
>> (modulo some language idiom changes) reflect the API that the C is
>> presenting to Ruby/PHP/Python/etc.
>>
> That's a really good question. One obvious factor is that the JNI bindings
> cover the full surface of the engine API, whereas the PHP and Ruby bindings
> only cover the messenger API, the latter being much smaller. The other
> factor is that we have a different workflow with how we maintain the
> scripting language bindings. From a design perspective each of those
> bindings are more tightly coupled to the C code since that is the only
> implementation they use, but from a workflow perspective each one is
> actually more independent than the Java binding. When we add a feature to
> the C messenger implementation we don't try to add it simultaneously across
> all the bindings. It generally goes into python because that is how we
> test, and it goes into perl and ruby pretty quickly after that because
> Darryl is good about keeping those up to date, but, e.g., in the case of
> php, we haven't tried to keep the binding at full feature parity. Finally,
> the JNI binding is a bit of a different animal from the others in that it
> is not just performing a simple wrapping of the C API and directly exposing
> that to Java. It is adapting between the C API underneath it and the Java
> API above it.
>
> As for the hastiness, perhaps that was a poor choice of words. The problem
> with the Java API/impl split is that the API was produced from an existing
> implementation after the fact in order to facilitate testing. As such it
> isn't well suited to be implemented by two independent implementations,
> particularly one that is JNI based.
>
>
>> To me the fact that APIs aren't defined but that instead the only way of
>> knowing what a particular call should do seems to be 1) look at the code in
>> C and, if it's not clear, then 2) ask Rafi; is why I no longer feel it
>> productive to try to contribute.  As such, it seems that this change is
>> more about vertical scaling (making it easier for Rafi, and those sitting
>> very near him, to work more productively), than horizontal scaling
>> (allowing people who are not Rafi or with easy access to him to work on
>> it).
>>
> Really there are two separate issues here. You're talking about conceptual
> barriers to contribution, and I'm talking about mechanical barriers. Both
> are barriers to contribution, and both prevent horizontal scaling. Fixing
> some of the mechanical barriers to contribution does happen to also improve
> vertical scaling, but as I said in my original post, I think the more
> important factor is making it easier/possible for people to make complete
> contributions.
>
> FWIW, I agree we need lots of additional work around API definition and
> documentation. Having more time for that is also one of the reasons I would
> like to reduce some of the overhead involved in the current development
> process.
>
>
>> Overall I don't care very much about Messenger in Java (either pure Java or
>> JNI), but I do care about the Engine.  When the Proton Engine was first
>> discussed it was always said that the C code should be able to be called
>> from within Java as an alternative to a pure Java impl. - has this changed?
>>
> I'm fine with that as a goal, however I can't say it's a priority for me.
> More importantly, as I've said before, I don't believe the use of JNI in
> production is at all compatible with the use of JNI for testing purposes.
> The current JNI binding exposes each part of the engine API in very fine
> detail. This what you want for testing, so you can exercise every part of
> the C API from Java. This is probably exactly the opposite of what you'd
> want for production use though. I expect you'd want to minimize the number
> of times you cross the C/Java boundary and just tie into the engine API at
> key points that are shifting a lot of bytes around.
>
>
>> Going back to the horizontal vs. vertical scaling... If the project were
>> structured differently I would be very happy to contribute to the engine
>> side of things in Java.  However I think that would require us to be
>> organised differently with a recognition that the Engine and Messenger are
>> conceptually separate (with Messenger depending on the Engine API, and
>> having no more authority over defining/testing the engine API than any
>> other client - such as the C++ broker, ActiveMQ, or the upcoming JMS clent,
>> etc.). As such I would rather see us fix the definition of the API (if you
>> believe it to be "hasty") rather than simply try to remove any notion of
>> their being an API which is distinct from the implementation.
>>
> I'm not sure exactly what you're asking for here with respect to Messenger
> and Engine. I believe they are currently layered precisely the way you
> describe. There certainly aren't any code level dependencies from Engine to
> Messenger. Are there specific ways that you think Messenger has influenced
> the Engine that are problematic? Is there something concrete you would like
> to see happen here?
>
> As for the Java API, I'm not suggesting eliminating the notion of there
> being an API. What I'm suggesting is that what sits in the proton-api
> module is not actually an API that is well suited for multiple
> implementations, but is really a single implementation API. As such I would
> like to put it back into the proton package and focus on making it a good
> single implementation API. I don't believe that this step would detract at
> all from it being a well defined API, in fact I think it would make it
> clearer and more transparent as there are currently plenty of cases where
> the split obscures the design. I believe a distinct question is whether we
> need a multi-implementation API, and if so for what purpose, e.g.
> production use of a JNI based engine vs testing of the C based engine.
>
> --Rafael
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message