qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rob Godfrey <rob.j.godf...@gmail.com>
Subject AMQP 1.0 Message Encoding - using AMQP typed data vs. Mime content-type / content-encoding (Was (unintentionally I assume): P)
Date Wed, 10 Sep 2014 17:08:40 GMT
On 10 September 2014 18:33, Alan Conway <aconway@redhat.com> wrote:

> On Tue, 2014-09-09 at 20:53 +0200, Rob Godfrey wrote:
> > On 9 September 2014 20:39, Fraser Adams <fraser.adams@blueyonder.co.uk>
> > wrote:
> >
> > > I'm afraid that your response was as clear as mud to me Rob, I'm
> having a
> > > "thick" day today :-(
> > >
> > > I did read the spec. and the RFCs, but frankly I work best with
> concrete
> > > examples.
> > >
> > > You say "
> > >
> > > the content type is (only) to be used when the data is being
> > > sent as an opaque "data" section and in that case it enables the
> recipient
> > > to interpret the opaque data correctly
> > >
> > > "
> > >
> > > That's interesting, and seems a bit overrestrictive.
> > >
> > >
> > So the rationale here is that if you put in "text/plain;
> charset="utf-16"'.
> > or something, then what does that mean?  The AMQP spec says that a string
> > is encoded in utf8...  As you said in you original mail, the AMQP type
> > system is supposed to be rich enough that it can be used to describe any
> > type.  The content type is designed for the case where the data being
> > transferred is not encoded in the AMQP type system, but instead presented
> > as opaque binary data in AMQP terms.
> >
> >
> > > As I say I'd quite like to use "application/json" so that I can use
> AMQP
> > > String but have the application interpret that as JSON, which seems
> like a
> > > perfectly reasonable use of the general concept of content-type and
> > > certainly how one would use content-type in something like HTTP (which
> I
> > > would have assumed that the AMQP 1.0 behaviour would most reasonably
> have
> > > been modelled after).
> > >
> > >
> > So - following the logic of the AMQP type system once could suppose that
> > the intended patern here would be to create a new described type (let's
> say
> > we use the description "org.apache.qpid.application/json") and use this
> to
> > send the string.
> >
> >
> > > If content-type isn't the correct place to pass information about the
> > > interpretation of the String I'm describing then I'd assume that
> > > annotations would be where I should go? Though I have to say that
> > > content-type=application/json seems by far and away the most natural
> place
> > > to describe the message content as being JSON.....
> > >
> > >
> > Why not send the "string" as opaque data then?  Basically using content
> > type and an AMQP typed value is mixing two different encoding systems...
> > Informally clients could probably support it but it's not really the
> > intended use case (the spec only says SHOULD - so it's not illegal, just
> > strongly discouraged :-) )
> >
>
> The problem with opaque data is that then you really have to set
> content-type='application/json; charset=<blah>' and supply your own
> unicode codec, which sucks since your friendly AMQP client library will
> deal with all that for you if you send an AMQP string message. So I can
> understand the desire to say "it's an AMQP string (so AMQP deals with
> unicode) but with an extra hint: that string is application/json". You
> could invent another property name for the hint but it really feels like
> it belongs in content-type.
>
>
I think it's inconsistent to use a mechanism (content type) intended for
opaque data on data which is not opaque... only in a couple of special
cases (an amqp-value section containing a string or binary value) does it
make sense.  In general the question is who is the field aimed at - the
AMQP library or the client application? I think the content-type should
actually be interpreted by the AMQP library - in the same way that I would
expect the AMQP types to be.

I think it "appears" to be more convenient to set the content-type on the
properties field for a string value because we perceive that the client
library would be able to understand AmqpValue(String("{ }")) but not
AmqpValue(Described(Symbol("org.json"),String("{ }"))) and that the
content-type would be made available to the end user of the API for them to
then be able to understand that this is Json, and not intercepted by the
client library.  However that's not necessarily the case (the current JMS
client doesn't expose the content type IIRC, and may use it for switching
behaviours on opaque data - e.g. for object messages).

More generally I think we collectively (in Qpid and in the wider AMQP
world) need to decide when/if we use the AMQP type system to encode data,
versus using MIME types.

If I want to start compressing data as I generally send large messages,
should I send my data with the content-encoding as gzip, or should I define
a new described type for compressed data.  If the former - what if the
large message I want to send is a map?

My general advice is to choose one mechanism or the other and not to
confuse the two.

Of course you have an inconsistency to resolve if somebody sets an AMQP
> string body with content-type='application/json; charset="utf-16"'
>
> Could we say: You MAY use content-type with an AMQP string body but it
> MUST NOT not have a charset and MUST be ignored by the sender. It MAY be
> used as a decoding hint on the receiver.
>
>
The spec is the spec, and now ratified at ISO, so the wording there is not
going to change.  We could interpret the spec more loosely, but - as above
- i think that ultimately that will lead to more confusion.

-- Rob


> > -- Rob
> >
> >
> > > Frase
> > >
> > >
> > > On 09/09/14 11:31, Rob Godfrey wrote:
> > >
> > >> Hi Frase,
> > >>
> > >> ignoring any transformations that may take place in a client library
> > >> between setting the content type in the API and what actually goes
> out on
> > >> the wire... the AMQP 1.0 specification defines content-type as follows
> > >> (see:
> > >> http://docs.oasis-open.org/amqp/core/v1.0/os/amqp-core-
> > >> messaging-v1.0-os.html#type-properties
> > >> ):
> > >>
> > >> The RFC-2046 [RFC2046
> > >>
> > >>> <http://docs.oasis-open.org/amqp/core/v1.0/os/amqp-core-
> > >>> overview-v1.0-os.html#anchor-RFC2046>]
> > >>> MIME type for the message's application-data section (body). As per
> > >>> RFC-2046 [RFC2046
> > >>> <http://docs.oasis-open.org/amqp/core/v1.0/os/amqp-core-
> > >>> overview-v1.0-os.html#anchor-RFC2046>]
> > >>> this can contain a charset parameter defining the character encoding
> > >>> used:
> > >>> e.g., 'text/plain; charset="utf-8"'.
> > >>> For clarity, as per section 7.2.1 of RFC-2616 [RFC2616
> > >>> <http://docs.oasis-open.org/amqp/core/v1.0/os/amqp-core-
> > >>> overview-v1.0-os.html#anchor-RFC2616>],
> > >>> where the content type is unknown the content-type SHOULD NOT be set.
> > >>> This
> > >>> allows the recipient the opportunity to determine the actual type.
> Where
> > >>> the section is known to be truly opaque binary data, the content-type
> > >>> SHOULD be set to application/octet-stream.
> > >>> When using an application-data section with a section code other than
> > >>> *data*, content-type SHOULD NOT be set.
> > >>>
> > >>
> > >> So, in essence the content type is (only) to be used when the data is
> > >> being
> > >> sent as an opaque "data" section and in that case it enables the
> recipient
> > >> to interpret the opaque data cofrrectly
> > >>
> > >> Hope this helps,
> > >> Rob
> > >>
> > >>
> > >>
> > >> On 8 September 2014 20:01, Fraser Adams <
> fraser.adams@blueyonder.co.uk>
> > >> wrote:
> > >>
> > >>  Gordon's recent post
> > >>>
> > >>> When the message is created it looks like this:
> > >>>
> > >>>       std::string messageContent("example text");
> > >>>       qpidMessage.setContentType("text/plain");
> > >>>       qpidMessage.setContent(qpid::types::Variant(messageContent));
> > >>>
> > >>> Use Message::setContentObject() instead and set the encoding, e.g.
> change
> > >>> the last line to:
> > >>>
> > >>>        qpidMessage.setContentObject(messageContent);
> > >>>        qpidMessage.getContentObject().setEncoding("utf8");
> > >>>
> > >>>
> > >>>
> > >>> prompted me to ponder about what and what is not the correct use of
> > >>> content-type in AMQP 1.0.
> > >>>
> > >>> So for AMQP 0.10 IIRC in qpid::messaging the content type was used
to
> > >>> indicate certain AMQP types, for example in particular amqp/list and
> > >>> amqp/map now as I understand it for AMQP 1.0 the AMQP type system is
> > >>> sufficiently self-describing such that maps/lists/whatever don't need
> > >>> such
> > >>> things as content-type, however the various APIs do need some
> mechanism
> > >>> to
> > >>> specify the relevant type hence (I think) the setContentObject()
> > >>> mechanism
> > >>> in qpid::messaging
> > >>>
> > >>> is that correct?
> > >>>
> > >>>
> > >>> Why I ask is that for my JavaScript stuff although I've incorporated
> a
> > >>> mechanism to automagically serialise and deserialise JavaScript
> Objects
> > >>> to
> > >>> the AMQP type system I also quite fancy including a mechanism that
> takes
> > >>> a
> > >>> JavaScript Object and serialises it as JSON in an AMQP String
> (obviously
> > >>> assuming that it is a data object that is actually convertable to
> JSON)
> > >>> similarly I'd like to be able to receive a JSON object and
> deserialise it
> > >>> into a JavaScript object.
> > >>>
> > >>> My thinking is that this is actually a good use of the AMQP
> content-type
> > >>> and in this case setting content-type to "application/json" is the
> right
> > >>> thing to do to give the API the right hints to treat Strings as JSON.
> > >>>
> > >>> Is my logic and understanding of the use of content-type in the AMQP
> 1.0
> > >>> spec. reasonable?
> > >>>
> > >>> Frase
> > >>>
> > >>>
> > >>>
> > >>>
> > >>>
> > >>> ---------------------------------------------------------------------
> > >>> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> > >>> For additional commands, e-mail: users-help@qpid.apache.org
> > >>>
> > >>>
> > >>>
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> > > For additional commands, e-mail: users-help@qpid.apache.org
> > >
> > >
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message