qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alan Conway <acon...@redhat.com>
Subject Re: AMQP 1.0 Message Encoding - using AMQP typed data vs. Mime content-type / content-encoding (Was (unintentionally I assume): P)
Date Mon, 22 Sep 2014 17:22:55 GMT
On Fri, 2014-09-12 at 08:27 +0100, Fraser Adams wrote:
> On 12/09/14 04:01, Alan Conway wrote:
> > On Thu, 2014-09-11 at 11:55 +0100, Gordon Sim wrote:
> >> On 09/10/2014 08:50 PM, Rob Godfrey wrote:
> >>> Part of the issue here, I think, is that we are not terribly (at all?)
> >>> clear on what each API might do with a particular input from the client
> >>> application, and how that might be treated by the library on the receiving
> >>> end.
> > I'm swayed by the arguments that we shouldn't mix content-type and AMQP
> > encoding on the wire. Fraser's original problem (send a string of
> > application/json) is not hard to handle with our existing APIs:
> >
> > In C++:
> >
> > std::string data = ...
> > m.setContentBytes(data)
> > m.setContentType("application/json")
> >
> > On the wire the C++ client uses a data section if you setContentBytes()
> > and an AMQP value section if you setContentObject().
> >
> > With Messenger this does the trick:
> >
> > m.put(Message(address='localhost/qq', body='xxx', inferred=True))
> >
> > Note the body must be of type str or bytes, not unicode. inferred=True
> > makes proton infer the section type from the body type, bytes is treated
> > as a data section. The default inferred=False always encodes an AMQP
> > value section, so even str or bytes would be encoded as an AMQP string.
> >
> 
> So can I replay some of this stuff to get it straight in my own mind?
> 
> What I'd *like* to be able to do in JavaScript is to do (at an API 
> level) something like
> 
> message.setAddress('amqp://localhost');
> message.setContentType('application/json');
> message.body = {"some": "json"};
> messenger.put(message);
> 
> 
> Without the setContentType this would be encoded by the JavaScript 
> binding into the AMQP type system as a Map containing a string named 
> "some" with a value of "json".
> 
> As an alternative I'd like to give the user the option of sending it as JSON
> 
> In that case the (internal) message._preEncode() call checks the content 
> type and if application/json instead of doing its call to
> 
>          body['putObject'](this['body']);
> 
> it invokes some code to do
> 
> |var bodyJSON = JSON.stringify(||this['body']);|
> 
> 
> And then send bodyJSON as an AMQP data section?
> 
> 
> 
> It's this last bit that I'm still not clear about in Messenger.
> 
> 
> I've already got a mechanism whereby I can send binary data (I created a 
> proton.Data.Binary class and can do fun stuff with ArrayBuffers). So I 
> can easily send a Binary child of the messenger's body object.
> 
> If I was sending a Binary normally I might do something like:
> 
> message.body = new proton.Data.Binary([65, 77, 81, 80]);
> 
> (The Binary constructor takes Array, ArrayBuffer, TypedArray, String) so 
> it's pretty trivial to do
> new proton.Data.Binary(|bodyJSON|);
> 
> To create a Binary containing the bytes of the JSON String.
> 
> The bit that confuses me (and this is just one bit of the Messenger API 
> that seems like black magic to me......)
> 
> 
> Doing what I've just done, if I understand correctly is *not enough* to 
> send as an AMQP data section, that Binary would still be encoded as a 
> value section?
> 
> 
> If so what would my Binary in this instance actually be decoded as by 
> say qpid::messaging or JMS? A BytesMessage, TextMessage, something else? 
> Or are both fairly tolerant when *decoding* data sections (TBH that 
> would be my guess)
> 
> 
> 
> Alan's comment "The default inferred=False always encodes an AMQP value 
> section, so even str or bytes would be encoded as an AMQP string ", 
> which suggests my Binary would be encoded as an AMQP string in a value 
> section on the wire? Is that correct?
> 
> TBH I'm not totally convinced, surely it's still encoded as a Binary - 
> albeit in a value section. 

You are be right there, I was not looking at the typecodes carefully
enough.

> As some corroboration to my assertion 
> something interesting I've observed - as you know I do a lot of QMF and 
> as it happens I get a bunch of QMF responses from qpidd and (flipping 
> annoyingly!!!) everything that *should* intuitively be a String gets 
> decoded as a Binary 'cause C++ strings generally get treated as AMQP 
> binaries by the C++ APIs (I believe that you have to be explicit with 
> the encoding) - just sayin', man that's annoying (it turns into 
> ClassCastException hell in Java 'cause it's not unreasonable to be 
> expecting to receive String, sigh!). So basically (at least for the case 
> of AMQP 0.10-> AMQP 1.0 translations) I've very definitely seeing AMQP 
> Binary values in value sections, and when I send a proton.Data.Binary I 
> very definitely call pn_data_put_binary.
> 
> 
> Anyway I digress.....
> 
> So if I actually want to send my Binary containing my stringified JSON 
> as a data section I have to do:
> 
> message.setInferred(true);
> 
> Is that correct?
> 
> 
> I think it is if I've understood Alan's post correctly.

That's what I observe with the python binding. Double checked these:

>>> msg.put(Message(address='0.0.0.0', body='foo')); msg.send() # Binary Value
>>> msg.put(Message(address='0.0.0.0', body='foo', inferred=True)); msg.send() #
Binary Data
>>> msg.put(Message(address='0.0.0.0', body=u'foo')); msg.send() # String Value
>>> msg.put(Message(address='0.0.0.0', body=u'foo', inferred=True)); msg.send() #
String Value

> But I'm left with one other thing that confuses me (sorry, it's early 
> :->), Alan said "Note the body must be of type str or bytes, not unicode 
> " now I understand the bytes, but less so the str. The documentation for 
> inferred says "If inferred is true then binary and list values in the 
> body of the message will be encoded as AMQP DATA and AMQP SEQUENCE 
> sections, respectively" but doesn't mention strings.

That is a python thing, not an AMQP or proton thing. In python 'unicode'
is the string type and 'str' is just an alias for 'bytes'. So python
binding treats str as "binary". So 'foo' is an object of types
str==bytes and will be treated as binary. u'foo' is an object of type
unicode and will be treated as a string.

> Now from what I can see at an API level bytes, string and symbol follow 
> a common pattern e.g.
> 
> pn_data_put_binary(data, pn_bytes(b.size, b.start));
> pn_data_put_string(data, pn_bytes(strlen(text), text));
> pn_data_put_symbol(data, pn_bytes(strlen(text), text));
> 
> Are you saying if I were to use put_binary or put_string with inferred 
> true they'd both be sent as data sections?

Nope, see above about confusion of 'string' vs. 'str', 'bytes' and
'unicode'.

>  That's not how I read the 
> documentation, but I could be wrong, perhaps it's some sugar of the 
> Python API, but TBH I thought that it behaved pretty much like the 
> JavaScript API and would call pn_data_put_string if the body was a string.
> 
> 
> Could you please clarify?

Yep, the confusion here comes from python. I think you are on the right
track.




---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org


Mime
View raw message