esme-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ethan Jewett <>
Subject Re: Metadata handling (was "Release planning")
Date Sun, 18 Jul 2010 05:59:34 GMT
On Sunday, July 18, 2010, Vassil Dichev <> wrote:
> Ethan, we should probably update the issue to properly reflect what
> you're saying- that the metadata shouldn't be encoded.

Agreed, I just didn't get around to it. Should be able to investigate
other uses of metadata and update the Jira issue this morning, unless
someone else gets there first.

> Note that
> URL-encoding is not an issue, cannot be avoided and is required for
> the Twitter API (as in the API of as well.

Yes, we require URL- or form- (I think) encoding on the API2 as well.
I meant that we would have to URL-encode the metadata before saving
the Message object, so that we get rid of the angle-brackets. Then the
Message will not think it is XML and throw away all the tags.

I think this is not the right way to do it anyway. Instead, I suspect
we should modify the internal handling of metadata, but I need to do
that research to make sure we don't break anything when/if we change


> I think this could be implemented reasonably easily, but handling
> would probably be different in XML and JSON representations. Let me
> look and I could recommend some way of doing it if someone wants to
> implement it.
> On Sat, Jul 17, 2010 at 9:08 PM, Ethan Jewett <> wrote:
>> Oh .... I see how it works. Thanks for the example. I was never able
>> to understand this before. This functionality is not what we want, but
>> just for completeness, here is a summary of how passing in metadata
>> through the API currently works (basic XML example):
>> Send in: <outer><meta><metameta>Hello</metameta></meta><onlymeta>Meta</onlymeta></outer>
>> Get out: HelloMeta
>> Note: this is the text contents of the XML tags concatenated together
>> in "order", whatever "order" means for XML content.
>> Here's a JSON example:
>> Send in: <anytag>"meta":[{"place":{"place_type":"city","region":"CA+"}},{"song":{"artist":"Prince","songtitle":"Never+Let+Me+Down"}}]</anytag>
>> Get out: &quot;meta&quot;:[{&quot;place&quot;:{&quot;place_type&quot;:&quot;city&quot;,&quot;region&quot;:&quot;CA+&quot;}},{&quot;song&quot;:{&quot;artist&quot;:&quot;Prince&quot;,&quot;songtitle&quot;:&quot;Never+Let+Me+Down&quot;}}]
>> Note: This is, again, the text contents of the XML tags. The output is
>> HTML entity encoded, as it was before, but there weren't any
>> characters in the previous example that would have been encoded.
>> Second note: The input *must* be wrapped in valid XML tags or it is ignored.
>> I believe that this is completely wrong. The API should simply take
>> the text assigned to the metadata parameter, store it as part of the
>> message, and return exactly the same string in the message response.
>> It should not care what is in the text and it should not modify it in
>> any way.
>> I think that making this work is going to take some revamping of the
>> internal handling of metadata, moving any kind of parsing and/or HTML
>> entity encoding to the UI code and have the internal Message model
>> simply store the metadata without thinking about it so hard. This
>> could clearly result in breakage, so I'm not sure how best to proceed.
>> We could just put code into the API to make this behave how we want by
>> URLencoding it wrapping it in a special XML tag before storing it as
>> part of the message, then un-entity encode it and then un-URLencode it
>> before sending it back to the client, but that is going to result in a
>> horrible mess stored in the actual message. I think it will be better
>> to change the way that the actual Message class handles metadata.
>> Does anyone know what functionality of the web UI currently uses
>> metadata, so that we can test for breakage if we change how this
>> works?
>> Imtiaz, if you want to work on making this behave in a sane way, that
>> would be fantastic. It's a big job, but we've got to start somewhere.
>> I think ESME-242 is a good umbrella issue for this and we can modify
>> the issue to better reflect the problem.
>> Ethan
>> On Sat, Jul 17, 2010 at 6:57 PM, Imtiaz Ahmed H E <> wrote:
>>> For example, for the response of the following (GET) maybe we need to url
>>> decode the response before we (the api) respond with it ? (Reference:  see
>>> the reponse to the POST of a message in my previous mail, forwarded below).
>>> curl -b headers
>>> http://localhost:8080/esme-server-apache-esme-1.0-RC1-incubating/api2/user/messages
>>> Of course this may be a typical naive one from me...! But let me know. Dick
>>> had said maybe I look at 242 before Vassil made his point & this discussion
>>> took place...
>>> Imtiaz
>>> ----- Original Message ----- From: "Imtiaz Ahmed H E" <>
>>> To: <>; "Ethan Jewett" <

View raw message