qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Fraser Adams <fraser.ad...@blueyonder.co.uk>
Subject Re: JavaScript version of Proton?
Date Fri, 03 Mar 2017 12:27:35 GMT
Hey Paul,

No worries, you didn't come across as being dismissive at all, it was 
more the comment about "pure-JavaScript" that I wanted to clarify - 
asm.js doesn't *require* specific support (other than a JavaScript 
runtime that supports ArrayBuffers) so although it's not exactly what 
you'd manually write yourself it *is* pure JavaScript that runs on 
anything that supports ArrayBuffers, which includes nodejs and most 
modern browsers. I guess I'm just being semantically pedantic on "pure 
JavaScript" I see what you mean when you say "written directly/natively" 
and for sure it's not that, though by the time you set about minifying 
most JS.....

Re "the benefits of cross-compiling C code to JavaScript has imho 2 big 
advantages: performance and not getting out of sync with the main (c) 
codebase." yeah that was my motivation, unfortunately it hasn't quite 
panned out how I was hoping so apologies for that. If I can find more 
time I may start playing with a reactor based one, but I'm spinning a 
few plates at the moment :-D.

I'd agree with your last paragraph. At work we've generally been using 
the C++ qpid::messaging and JMS clients, so we've been a bit isolated 
from the variety of proton APIs. I'm with you on the polyglot stuff and 
I'd really like to see a bit more alignment across the various language 
bindings. Obviously some things are harder to do - for example doing a 
JMS like API for JS would probably be more trouble than it's worth but 
having a consistent high-ish level API for a range of modern languages 
that supports the sort of high-level link-attachment stuff that 
qpid::messaging and JMS support would be handy.

As I see it one really useful direction for AMQP and proton would be for 
providing AMQP endpoints on cloud based microservices. AMQP 1.0 is more 
a peer-to-peer protocol and doesn't necessarily require a broker and 
having a microservice expose its own AMQP 1.0 endpoint in addition to an 
HTTP endpoint would be great for exposing asynchronous microservices. As 
I say having a good (and consistent) high level API would I think help 
promote a polyglot approach so I can use 
C++/Python/Go/Node/Java/whatever wherever most appropriate.

I'd hate to leave you with too negative a view, I think it's mostly a 
case of different people working on it to different priorities. I just 
wanted to give you an honest answer from my own perspective and if I've 
misinterpreted anything I'll let the others correct me.

Cheers,

Frase


On 03/03/17 11:22, Paul wrote:
> Hi Frase,
>
> Sorry if I came off as dismissing the Emscripten approach: I was just 
> looking for something written directly/natively in JavaScript instead 
> of cross-compiled C code, based on how things were worded on the website.
>
> And I'm also not saying that one is better than the other: the 
> benefits of cross-compiling C code to JavaScript has imho 2 big 
> advantages: performance and not getting out of sync with the main (c) 
> codebase.
>
> Based on what you wrote, I sense that the C and Java API are not in 
> sync either? Based on the wording on the website (that has since been 
> removed) I thought that all implementations (C, Java and JavaScript) 
> where being automatically testing using the same testsuite, to make 
> sure they all are in line with each other. As that has since been 
> removed from the website and according to you the Java and C API have 
> diverged, I conclude that there is no single set of tests with which 
> all implementations are tested automatically?
>
> The reason I got interested in Proton is that we use both Java and 
> JavaScript (Node) in our technology stack and for both we need an AMQP 
> client, so the idea of having the same library (natively) implemented 
> in both languages with the same API sounded very interesting. But from 
> what I gather from the answers so far, this isn't the case, which is 
> too bad and makes Qpid Proton loose one of pluses for me.
>
> Paul
>
>
> On 3/3/2017 11:04 AM, Fraser Adams wrote:
>> Hello Paul,
>>
>> I contributed the emscripten based proton JavaScript binding a while 
>> back.
>>
>> 1. Re: "The only think I can find are JavaScript bindings that are 
>> cross-compiled from the C version using Emscripten:
>> https://github.com/apache/qpid-proton/tree/master/proton-c/bindings/javascript 
>>
>>
>> That doesn't look like the 'pure-JavaScript implementation' mentioned 
>> on http://qpid.apache.org/proton/""
>>
>> I think that's a little unfair, if you take a look at what emscripten 
>> is it is an LLVM back-end that allows things supported by LLVM 
>> front-ends such as C/C++ etc. to be compiled to that back-end 
>> language, so in practice it compiles the proton C code into asm.js 
>> http://asmjs.org/faq.html which *is* pure JavaScript and certainly 
>> closer to pure JavaScript than most of the node bindings I've seen 
>> which compile the C code and use native code bindings so won't run in 
>> a browser. The original intent of the JavaScript binding was to have 
>> something that would run in a browser and as it happens also ran 
>> happily in node.
>>
>> 2. It was built to use the proton messenger API. I suspect that's 
>> probably the main issue now, as many people seem to have moved away 
>> from messenger due to issues with error handling when the remote end 
>> terminated. Funnily enough though, the JavaScript binding was 
>> actually more tolerant than the basic C one because I actually monkey 
>> patched this stuff so when I received websocket error events the 
>> JavaScript binding reconnected fairly well. I tried to make the 
>> JavaScript binding to messenger follow the same patterns as the 
>> python binding, so it had a fairly idiomatic feel JavaScript feel.
>>
>> 3. "I had a brief look at the JavaScript bindings history and while 
>> it does seem to get an occassional update, I wonder if it is up to 
>> date with the ongoing developments of Qpid Proton ". That's a fair 
>> observation. It actually took quite a bit of energy to put the 
>> binding together in the first place, but it was entirely my spare 
>> time effort and I'm afraid I've been busy on other things recently. 
>> TBH there didn't seem to be a great deal of interest in it, so I'm 
>> afraid I somewhat lost motivation.
>>
>> I think that when Gordon did rhea https://github.com/grs/rhea I got 
>> fairly confused about what the direction was going to be, especially 
>> when the dispatch router UI started using that instead of the 
>> emscripten based binding (as I say there didn't seem to be a great 
>> deal of interest which had a bit of an effect on my motivation). I 
>> realise that the emscripten approach can result in somewhat larger 
>> javascript files, but OTOH because it is based on a virtual heap 
>> built on ArrayBuffers it had pretty good support for being able to 
>> transport arbitrary binary objects, which is something that many 
>> JavaScript implementations of things often fall down on, I'm not sure 
>> if rhea supports this (it may, I've not checked) but I explicitly did 
>> that for the emscripten based binding, which works fairly well as 
>> it's built on ArrayBuffers.
>>
>> I keep meaning to get back to it, I think a lot of the thinking in 
>> proton has been around reactive APIs which are a better fit for 
>> JavaScript and it'd make sense to move away from messenger API to 
>> those, but as I say in general there hasn't been that much interest 
>> and I've had commitments elsewhere.
>>
>> 4. Re "In fact, from a recent mail thread I think it may not even 
>> compile at present, and its probably overdue for the community to 
>> have a discussion about its future as a result. ". That's a shame, 
>> TBH one of the problems I had was chasing my tail keeping up to date 
>> with changes given juggling this with family and other tech 
>> commitments, I had sort of hoped that the community would keep an eye 
>> on it and at least keep it compiling.
>>
>>
>> It's a difficult call, in actual fact one of the motivations for 
>> going down the emscripten based approach was because it is 
>> essentially a case of compiling from the "canonical" proton C code. 
>> It really *should* be possible to be slap bang up to date with 
>> whatever is in the underlying C trunk (provided of course a little 
>> TLC is given). Part of the reason I went with that approach was that 
>> I'd seen proton-C and proton-J diverge and I figured that if I'd done 
>> a ground up JavaScript implementation it would stand less chance of 
>> keeping up with changes and improvements to the underlying protocol 
>> engine and the code base would diverge. I still think that is 
>> actually a fair premise and conceptually follows the approach of the 
>> various swigged bindings, but as I say is still needs a little TLC to 
>> make that situation hold.
>>
>> Another reason why emscripten can be a good choice is because of 
>> performance (a key use case for emscriptem is porting games to the 
>> web - take a look at the website there are some really cool things). 
>> To be fair I haven't compared the proton JS binding performance with 
>> rhea, but in the general sense compiling well written C/C++ 
>> applications to JavaScript can give much better performance than pure 
>> idiomatic JavaScript partly because of things like type coercion, 
>> partly because of the virtual heap and avoiding garbage collection 
>> lags and partly because some engines (notably in FireFox and Edge) 
>> have really good asm.js optimisations and do ahead-of-time 
>> compilation of asm.js blocks. When I emscriptened gzip in FireFox a 
>> while back I was seeing > 80% native performance and it's improving 
>> all the time. There is also a roadmap towards WebAssembly which 
>> should increase the performance even more. The down side is of course 
>> the larger JS file and some awkwardness binding idiomatic JavaScript 
>> to a C-like API. Most of the actual JS code in the proton JavaScript 
>> binding is the "binding" stuff to make it look like idiomatic 
>> JavaScript. If you take a look at that stuff and compare it with the 
>> python binding that wraps the swigged C you should see that they are 
>> actually pretty similar (I actually used the python binding for 
>> inspiration TBH).
>>
>> I think that the arguments for and against doing an emscriptened 
>> binding and a ground up implementation are largely the same as those 
>> for using a swig binding versus a ground up implementation for other 
>> languages.
>>
>>
>> Sorry it's not a really "tidy" answer, but at least it's an honest 
>> one :-)
>>
>> I'm hoping to find time to get back into doing qpid things, and 
>> JavaScript is one of the things I care most about, but I'd definitely 
>> like to see somewhat more cohesion. By my observation Proton has got 
>> a bit "fragmented" with a range of APIs and not seen a lot of clarity 
>> on approach/roadmap and it has seemed that different languages seem 
>> to have taken different approaches so I've never really pinned down 
>> what might be described as a common API. On the one hand so what, but 
>> on the other I'm a big fan of polyglot software development and it'd 
>> be quite nice to be able to switch between languages and follow a 
>> somewhat similar approach, which doesn't seem to be the case at the 
>> moment.
>>
>> Another thing that seems to be missing (though I may just be out of 
>> the loop) is a clean way to use the AMQP link attachment protocol, 
>> for example if you look at the qpid messaging API (that's the one in 
>> the main qpid repo) and also JMS they make use of a JSON-like 
>> "address string" which contains a "link: " stanza that allows one to 
>> configure how link attachment works, that's really useful for AMQP 
>> consumers subscribing to messages off a broker as you can use that to 
>> specify message selectors and do useful things like have 
>> non-exclusive queues where multiple instances of a consumer can read 
>> messages off the same queue, which is great if you want to scale out 
>> consumers. Now you can do that with the proton API (and indeed 
>> messenger uses the low-level proton engine), but it's quite fiddly 
>> and it would be nice to see APIs to make that much more convenient.
>>
>> It'd also be nice to have a node binding that accepted both 
>> WebSockets and TCP, of course browsers can only do the former and 
>> that was the main target of the emscripten based binding so you could 
>> certainly create a node app that you could connect to via a WebSocket 
>> from a browser, but I never got round to supporting TCP connections 
>> too which would be useful. The Java broker supports both which is 
>> really helpful but the C++ broker (which is the one I mainly use) 
>> does not (which would be a nice addition). I *think* that there is 
>> work ongoing in dispatch router to support WebSockets and that may be 
>> a good way to bridge to the C++ broker but it'd still be nice if it 
>> had native support.
>>
>> HTH,
>>
>> Regards,
>>
>> Frase
>>
>>
>>
>>
>> On 03/03/17 07:51, Paul wrote:
>>> Tnx for the claification.
>>>
>>> Too bad the things claimed on the website aren't true, as I was 
>>> particular interested in being able to use the exact same 
>>> library/API in both Java and JavaScript (as we use both in our 
>>> stack), as to not have to deal with two separate libraries/api's 
>>> that do the same thing, albeit in a different language.
>>>
>>> Paul
>>>
>>> On 3/2/2017 11:46 AM, Robbie Gemmell wrote:
>>>> On 2 March 2017 at 07:56, Paul <pgbakker@gmail.com> wrote:
>>>>> Hi,
>>>>>
>>>>> Based on http://qpid.apache.org/proton/overview.html (and
>>>>> http://qpid.apache.org/proton/), I'm looking for the JavaScript
>>>>> implementation.
>>>>>
>>>>> The only think I can find are JavaScript bindings that are 
>>>>> cross-compiled
>>>>> from the C version using Emscripten:
>>>>> https://github.com/apache/qpid-proton/tree/master/proton-c/bindings/javascript

>>>>>
>>>>>
>>>>> That doesn't look like the 'pure-JavaScript implementation' 
>>>>> mentioned on
>>>>> http://qpid.apache.org/proton/ and the 'At its core Proton 
>>>>> provides three
>>>>> parallel implementations.....' mentioned on
>>>>> http://qpid.apache.org/proton/overview.html
>>>>>
>>>>> Am I completely overlooking something somewhere or?
>>>>>
>>>> No, you aren't missing anything, the site is in error.
>>>>
>>>> I think these references are the result of folks getting a little
>>>> ahead of reality in times past and it not subsequently being spotted
>>>> and reigned back in to reflect what actually happened, which was the
>>>> binding using emscripten that you referenced. I have updated the site
>>>> to remove those mentions, thanks for pointing them out.
>>>>
>>>>> I had a brief look at the JavaScript bindings history and while it 
>>>>> does seem
>>>>> to get an occassional update, I wonder if it is up to date with 
>>>>> the ongoing
>>>>> developments of Qpid Proton
>>>> I think it would be fair to say the JavaScript binding is not up to
>>>> date with ongoing development. In fact, from a recent mail thread I
>>>> think it may not even compile at present, and its probably overdue for
>>>> the community to have a discussion about its future as a result.
>>>>
>>>> For now, if you are looking for a Javascript AMQP 1.0 implementation
>>>> you might consider one written by Gordon Sim,
>>>> https://github.com/grs/rhea, which loosely follows the style of the
>>>> Proton Container api in the Python binding which Gordon also did much
>>>> of the work on.
>>>>
>>>>> TIA,
>>>>>
>>>>> Paul
>>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
>>>> For additional commands, e-mail: users-help@qpid.apache.org
>>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
>>> For additional commands, e-mail: users-help@qpid.apache.org
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
>> For additional commands, e-mail: users-help@qpid.apache.org
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org


Mime
View raw message