Mailing-List: contact users-help@qpid.apache.org; run by ezmlm
Precedence: bulk
Reply-To: users@qpid.apache.org
MIME-Version: 1.0
In-Reply-To: <cc298e10-09f7-fd30-3a87-c29d1578bdc4@gmail.com>
References: <331c223f-01a1-26f7-d133-35d09807bb31@gmail.com>
 <CAFitrpSP55+itPw36Xe_qOqw6zo5DC6pQi35T6Xv9mJs9HnRhw@mail.gmail.com>
 <f12c068e-0904-0b3a-5af2-6d7d6bd943a1@gmail.com> <29b5a5a3-edba-2e15-18b2-30d1e649aac6@blueyonder.co.uk>
 <cc298e10-09f7-fd30-3a87-c29d1578bdc4@gmail.com>
From: Robbie Gemmell <robbie.gemmell@gmail.com>
Date: Fri, 3 Mar 2017 12:24:38 +0000
Message-ID: <CAFitrpRzpOUbNWd76m1u=NnfLV2QN3vs2PBSeGH6hMVdmaUvQw@mail.gmail.com>
Subject: Re: JavaScript version of Proton?
To: "users@qpid.apache.org" <users@qpid.apache.org>
Content-Type: text/plain; charset=UTF-8
archived-at: Fri, 03 Mar 2017 12:24:49 -0000

On 3 March 2017 at 11:22, Paul <pgbakker@gmail.com> wrote:
> Hi Frase,
>
> Sorry if I came off as dismissing the Emscripten approach: I was just
> looking for something written directly/natively in JavaScript instead of
> cross-compiled C code, based on how things were worded on the website.
>
> And I'm also not saying that one is better than the other: the benefits of
> cross-compiling C code to JavaScript has imho 2 big advantages: performance
> and not getting out of sync with the main (c) codebase.
>
> Based on what you wrote, I sense that the C and Java API are not in sync
> either? Based on the wording on the website (that has since been removed) I
> thought that all implementations (C, Java and JavaScript) where being
> automatically testing using the same testsuite, to make sure they all are in
> line with each other. As that has since been removed from the website and
> according to you the Java and C API have diverged, I conclude that there is
> no single set of tests with which all implementations are tested
> automatically?
>

There is a long and complicated history to how things started out with
Proton, then evolved beyond its original scope as a protocol engine,
and got to where they are now. I'd need a mail longer than Frasers
(which I still need to actually read) to really fully explain it all.

Short version, there are in effect multiple APIs within the scope of
Proton at this point targetting different uses, where some of them
spread across the languages[/bindings], other have not [yet], and some
have even spread and since been removed (in the case of Messenger,
which was removed from proton-j recently). The core protocol engine
API, the original goal for Proton, is still very similar between
Proton-C and Proton-J (arguably too similar in cases) but this is a
fairly low level API intended and used for integrations in
clients/brokers/routers/etc, and so not necessarily what many are
looking for directly. The higher level APIs since built on top of this
are where most of the differences actually come out.

Proton-C (and its various bindings, such as Javascript) and Proton-J
lived in the same repo until January when we moved Proton-J out to its
own repo to be independent going forward. They shared a common test
suite written in Python based around the python bindings and adapter
shims, though some tests were specific to proton-c in cases it had
diverged, mostly cases where it made very good sense to do so as the
needs/behaviours of the languages are entirely different. Proton-J
still has its own copy of [a subset] of this test suite in its build,
for now.

> The reason I got interested in Proton is that we use both Java and
> JavaScript (Node) in our technology stack and for both we need an AMQP
> client, so the idea of having the same library (natively) implemented in
> both languages with the same API sounded very interesting. But from what I
> gather from the answers so far, this isn't the case, which is too bad and
> makes Qpid Proton loose one of pluses for me.
>
> Paul
>
>
>
> On 3/3/2017 11:04 AM, Fraser Adams wrote:
>>
>> Hello Paul,
>>
>> I contributed the emscripten based proton JavaScript binding a while back.
>>
>> 1. Re: "The only think I can find are JavaScript bindings that are
>> cross-compiled from the C version using Emscripten:
>>
>> https://github.com/apache/qpid-proton/tree/master/proton-c/bindings/javascript
>>
>> That doesn't look like the 'pure-JavaScript implementation' mentioned on
>> http://qpid.apache.org/proton/""
>>
>> I think that's a little unfair, if you take a look at what emscripten is
>> it is an LLVM back-end that allows things supported by LLVM front-ends such
>> as C/C++ etc. to be compiled to that back-end language, so in practice it
>> compiles the proton C code into asm.js http://asmjs.org/faq.html which *is*
>> pure JavaScript and certainly closer to pure JavaScript than most of the
>> node bindings I've seen which compile the C code and use native code
>> bindings so won't run in a browser. The original intent of the JavaScript
>> binding was to have something that would run in a browser and as it happens
>> also ran happily in node.
>>
>> 2. It was built to use the proton messenger API. I suspect that's probably
>> the main issue now, as many people seem to have moved away from messenger
>> due to issues with error handling when the remote end terminated. Funnily
>> enough though, the JavaScript binding was actually more tolerant than the
>> basic C one because I actually monkey patched this stuff so when I received
>> websocket error events the JavaScript binding reconnected fairly well. I
>> tried to make the JavaScript binding to messenger follow the same patterns
>> as the python binding, so it had a fairly idiomatic feel JavaScript feel.
>>
>> 3. "I had a brief look at the JavaScript bindings history and while it
>> does seem to get an occassional update, I wonder if it is up to date with
>> the ongoing developments of Qpid Proton ". That's a fair observation. It
>> actually took quite a bit of energy to put the binding together in the first
>> place, but it was entirely my spare time effort and I'm afraid I've been
>> busy on other things recently. TBH there didn't seem to be a great deal of
>> interest in it, so I'm afraid I somewhat lost motivation.
>>
>> I think that when Gordon did rhea https://github.com/grs/rhea I got fairly
>> confused about what the direction was going to be, especially when the
>> dispatch router UI started using that instead of the emscripten based
>> binding (as I say there didn't seem to be a great deal of interest which had
>> a bit of an effect on my motivation). I realise that the emscripten approach
>> can result in somewhat larger javascript files, but OTOH because it is based
>> on a virtual heap built on ArrayBuffers it had pretty good support for being
>> able to transport arbitrary binary objects, which is something that many
>> JavaScript implementations of things often fall down on, I'm not sure if
>> rhea supports this (it may, I've not checked) but I explicitly did that for
>> the emscripten based binding, which works fairly well as it's built on
>> ArrayBuffers.
>>
>> I keep meaning to get back to it, I think a lot of the thinking in proton
>> has been around reactive APIs which are a better fit for JavaScript and it'd
>> make sense to move away from messenger API to those, but as I say in general
>> there hasn't been that much interest and I've had commitments elsewhere.
>>
>> 4. Re "In fact, from a recent mail thread I think it may not even compile
>> at present, and its probably overdue for the community to have a discussion
>> about its future as a result. ". That's a shame, TBH one of the problems I
>> had was chasing my tail keeping up to date with changes given juggling this
>> with family and other tech commitments, I had sort of hoped that the
>> community would keep an eye on it and at least keep it compiling.
>>
>>
>> It's a difficult call, in actual fact one of the motivations for going
>> down the emscripten based approach was because it is essentially a case of
>> compiling from the "canonical" proton C code. It really *should* be possible
>> to be slap bang up to date with whatever is in the underlying C trunk
>> (provided of course a little TLC is given). Part of the reason I went with
>> that approach was that I'd seen proton-C and proton-J diverge and I figured
>> that if I'd done a ground up JavaScript implementation it would stand less
>> chance of keeping up with changes and improvements to the underlying
>> protocol engine and the code base would diverge. I still think that is
>> actually a fair premise and conceptually follows the approach of the various
>> swigged bindings, but as I say is still needs a little TLC to make that
>> situation hold.
>>
>> Another reason why emscripten can be a good choice is because of
>> performance (a key use case for emscriptem is porting games to the web -
>> take a look at the website there are some really cool things). To be fair I
>> haven't compared the proton JS binding performance with rhea, but in the
>> general sense compiling well written C/C++ applications to JavaScript can
>> give much better performance than pure idiomatic JavaScript partly because
>> of things like type coercion, partly because of the virtual heap and
>> avoiding garbage collection lags and partly because some engines (notably in
>> FireFox and Edge) have really good asm.js optimisations and do ahead-of-time
>> compilation of asm.js blocks. When I emscriptened gzip in FireFox a while
>> back I was seeing > 80% native performance and it's improving all the time.
>> There is also a roadmap towards WebAssembly which should increase the
>> performance even more. The down side is of course the larger JS file and
>> some awkwardness binding idiomatic JavaScript to a C-like API. Most of the
>> actual JS code in the proton JavaScript binding is the "binding" stuff to
>> make it look like idiomatic JavaScript. If you take a look at that stuff and
>> compare it with the python binding that wraps the swigged C you should see
>> that they are actually pretty similar (I actually used the python binding
>> for inspiration TBH).
>>
>> I think that the arguments for and against doing an emscriptened binding
>> and a ground up implementation are largely the same as those for using a
>> swig binding versus a ground up implementation for other languages.
>>
>>
>> Sorry it's not a really "tidy" answer, but at least it's an honest one :-)
>>
>> I'm hoping to find time to get back into doing qpid things, and JavaScript
>> is one of the things I care most about, but I'd definitely like to see
>> somewhat more cohesion. By my observation Proton has got a bit "fragmented"
>> with a range of APIs and not seen a lot of clarity on approach/roadmap and
>> it has seemed that different languages seem to have taken different
>> approaches so I've never really pinned down what might be described as a
>> common API. On the one hand so what, but on the other I'm a big fan of
>> polyglot software development and it'd be quite nice to be able to switch
>> between languages and follow a somewhat similar approach, which doesn't seem
>> to be the case at the moment.
>>
>> Another thing that seems to be missing (though I may just be out of the
>> loop) is a clean way to use the AMQP link attachment protocol, for example
>> if you look at the qpid messaging API (that's the one in the main qpid repo)
>> and also JMS they make use of a JSON-like "address string" which contains a
>> "link: " stanza that allows one to configure how link attachment works,
>> that's really useful for AMQP consumers subscribing to messages off a broker
>> as you can use that to specify message selectors and do useful things like
>> have non-exclusive queues where multiple instances of a consumer can read
>> messages off the same queue, which is great if you want to scale out
>> consumers. Now you can do that with the proton API (and indeed messenger
>> uses the low-level proton engine), but it's quite fiddly and it would be
>> nice to see APIs to make that much more convenient.
>>
>> It'd also be nice to have a node binding that accepted both WebSockets and
>> TCP, of course browsers can only do the former and that was the main target
>> of the emscripten based binding so you could certainly create a node app
>> that you could connect to via a WebSocket from a browser, but I never got
>> round to supporting TCP connections too which would be useful. The Java
>> broker supports both which is really helpful but the C++ broker (which is
>> the one I mainly use) does not (which would be a nice addition). I *think*
>> that there is work ongoing in dispatch router to support WebSockets and that
>> may be a good way to bridge to the C++ broker but it'd still be nice if it
>> had native support.
>>
>> HTH,
>>
>> Regards,
>>
>> Frase
>>
>>
>>
>>
>> On 03/03/17 07:51, Paul wrote:
>>>
>>> Tnx for the claification.
>>>
>>> Too bad the things claimed on the website aren't true, as I was
>>> particular interested in being able to use the exact same library/API in
>>> both Java and JavaScript (as we use both in our stack), as to not have to
>>> deal with two separate libraries/api's that do the same thing, albeit in a
>>> different language.
>>>
>>> Paul
>>>
>>> On 3/2/2017 11:46 AM, Robbie Gemmell wrote:
>>>>
>>>> On 2 March 2017 at 07:56, Paul <pgbakker@gmail.com> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> Based on http://qpid.apache.org/proton/overview.html (and
>>>>> http://qpid.apache.org/proton/), I'm looking for the JavaScript
>>>>> implementation.
>>>>>
>>>>> The only think I can find are JavaScript bindings that are
>>>>> cross-compiled
>>>>> from the C version using Emscripten:
>>>>>
>>>>> https://github.com/apache/qpid-proton/tree/master/proton-c/bindings/javascript
>>>>>
>>>>> That doesn't look like the 'pure-JavaScript implementation' mentioned
>>>>> on
>>>>> http://qpid.apache.org/proton/ and the 'At its core Proton provides
>>>>> three
>>>>> parallel implementations.....' mentioned on
>>>>> http://qpid.apache.org/proton/overview.html
>>>>>
>>>>> Am I completely overlooking something somewhere or?
>>>>>
>>>> No, you aren't missing anything, the site is in error.
>>>>
>>>> I think these references are the result of folks getting a little
>>>> ahead of reality in times past and it not subsequently being spotted
>>>> and reigned back in to reflect what actually happened, which was the
>>>> binding using emscripten that you referenced. I have updated the site
>>>> to remove those mentions, thanks for pointing them out.
>>>>
>>>>> I had a brief look at the JavaScript bindings history and while it does
>>>>> seem
>>>>> to get an occassional update, I wonder if it is up to date with the
>>>>> ongoing
>>>>> developments of Qpid Proton
>>>>
>>>> I think it would be fair to say the JavaScript binding is not up to
>>>> date with ongoing development. In fact, from a recent mail thread I
>>>> think it may not even compile at present, and its probably overdue for
>>>> the community to have a discussion about its future as a result.
>>>>
>>>> For now, if you are looking for a Javascript AMQP 1.0 implementation
>>>> you might consider one written by Gordon Sim,
>>>> https://github.com/grs/rhea, which loosely follows the style of the
>>>> Proton Container api in the Python binding which Gordon also did much
>>>> of the work on.
>>>>
>>>>> TIA,
>>>>>
>>>>> Paul
>>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
>>>> For additional commands, e-mail: users-help@qpid.apache.org
>>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
>>> For additional commands, e-mail: users-help@qpid.apache.org
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
>> For additional commands, e-mail: users-help@qpid.apache.org
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org