incubator-esme-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From in.imt...@gmail.com
Subject Re: Metadata handling (was "Release planning")
Date Sun, 18 Jul 2010 05:42:17 GMT
Vassil said,
"I could recommend some way of doing it if someone wants to
implement it.".
Vassil, do, do so...
Ethan, waiting for this and for someone to update the Jira issue...
Imtiaz

Sent from BlackBerry® on Airtel

-----Original Message-----
From: Vassil Dichev <vdichev@apache.org>
Sender: vdichev@gmail.com
Date: Sun, 18 Jul 2010 07:34:57 
To: <esme-dev@incubator.apache.org>
Reply-To: esme-dev@incubator.apache.org
Subject: Re: Metadata handling (was "Release planning")

Ethan, we should probably update the issue to properly reflect what
you're saying- that the metadata shouldn't be encoded. Note that
URL-encoding is not an issue, cannot be avoided and is required for
the Twitter API (as in the API of twitter.com) as well.

I think this could be implemented reasonably easily, but handling
would probably be different in XML and JSON representations. Let me
look and I could recommend some way of doing it if someone wants to
implement it.


On Sat, Jul 17, 2010 at 9:08 PM, Ethan Jewett <esjewett@gmail.com> wrote:
> Oh .... I see how it works. Thanks for the example. I was never able
> to understand this before. This functionality is not what we want, but
> just for completeness, here is a summary of how passing in metadata
> through the API currently works (basic XML example):
>
> Send in: <outer><meta><metameta>Hello</metameta></meta><onlymeta>Meta</onlymeta></outer>
>
> Get out: HelloMeta
>
> Note: this is the text contents of the XML tags concatenated together
> in "order", whatever "order" means for XML content.
>
> Here's a JSON example:
>
> Send in: <anytag>"meta":[{"place":{"place_type":"city","region":"CA+"}},{"song":{"artist":"Prince","songtitle":"Never+Let+Me+Down"}}]</anytag>
>
> Get out: &quot;meta&quot;:[{&quot;place&quot;:{&quot;place_type&quot;:&quot;city&quot;,&quot;region&quot;:&quot;CA+&quot;}},{&quot;song&quot;:{&quot;artist&quot;:&quot;Prince&quot;,&quot;songtitle&quot;:&quot;Never+Let+Me+Down&quot;}}]
>
> Note: This is, again, the text contents of the XML tags. The output is
> HTML entity encoded, as it was before, but there weren't any
> characters in the previous example that would have been encoded.
>
> Second note: The input *must* be wrapped in valid XML tags or it is ignored.
>
> I believe that this is completely wrong. The API should simply take
> the text assigned to the metadata parameter, store it as part of the
> message, and return exactly the same string in the message response.
> It should not care what is in the text and it should not modify it in
> any way.
>
> I think that making this work is going to take some revamping of the
> internal handling of metadata, moving any kind of parsing and/or HTML
> entity encoding to the UI code and have the internal Message model
> simply store the metadata without thinking about it so hard. This
> could clearly result in breakage, so I'm not sure how best to proceed.
>
> We could just put code into the API to make this behave how we want by
> URLencoding it wrapping it in a special XML tag before storing it as
> part of the message, then un-entity encode it and then un-URLencode it
> before sending it back to the client, but that is going to result in a
> horrible mess stored in the actual message. I think it will be better
> to change the way that the actual Message class handles metadata.
>
> Does anyone know what functionality of the web UI currently uses
> metadata, so that we can test for breakage if we change how this
> works?
>
> Imtiaz, if you want to work on making this behave in a sane way, that
> would be fantastic. It's a big job, but we've got to start somewhere.
> I think ESME-242 is a good umbrella issue for this and we can modify
> the issue to better reflect the problem.
>
> Ethan
>
> On Sat, Jul 17, 2010 at 6:57 PM, Imtiaz Ahmed H E <in.imtiaz@gmail.com> wrote:
>> For example, for the response of the following (GET) maybe we need to url
>> decode the response before we (the api) respond with it ? (Reference:  see
>> the reponse to the POST of a message in my previous mail, forwarded below).
>>
>> curl -b headers
>> http://localhost:8080/esme-server-apache-esme-1.0-RC1-incubating/api2/user/messages
>>
>> Of course this may be a typical naive one from me...! But let me know. Dick
>> had said maybe I look at 242 before Vassil made his point & this discussion
>> took place...
>>
>> Imtiaz
>>
>> ----- Original Message ----- From: "Imtiaz Ahmed H E" <in.imtiaz@gmail.com>
>> To: <esme-dev@incubator.apache.org>; "Ethan Jewett" <esjewett@gmail.com>
>> Sent: Saturday, July 17, 2010 9:35 PM
>> Subject: Re: Release planning
>>
>>
>>> Well...here you are...
>>>
>>> imtiaz@imtiaz-20100131 /cygdrive/d/temp
>>> $ curl --dump-header headers -d "token=HEZTQKM525SAMIPN4EDVRUOGHI40AKBL"
>>> http:/
>>> /localhost:8080/esme-server-apache-esme-1.0-RC1-incubating/api2/session
>>> <?xml version="1.0" encoding="UTF-8"?>
>>>
>>> <api><session><user><id>3</id><nickname>imtiaz2</nickname><image>None</image><wh
>>> ole_name>I A 2 H E</whole_name></user></session></api>
>>>
>>> imtiaz@imtiaz-20100131 /cygdrive/d/temp
>>> $ cat headers
>>> HTTP/1.1 200 OK
>>> Server: Apache-Coyote/1.1
>>> Set-Cookie: JSESSIONID=28DBA5607A4D94C9E6E2BD1E61492EAD;
>>> Path=/esme-server-apach
>>> e-esme-1.0-RC1-incubating
>>> Expires: Sat, 17 Jul 2010 15:58:22 UTC
>>> Date: Sat, 17 Jul 2010 15:58:22 GMT
>>> Pragma: no-cache
>>> Cache-Control: no-cache; private; no-store
>>> X-Lift-Version: 2.0-SNAPSHOT
>>> Content-Type: text/xml;charset=utf-8
>>> Content-Length: 178
>>>
>>>
>>> imtiaz@imtiaz-20100131 /cygdrive/d/temp
>>> $ curl -b headers -d
>>> 'message=test72&metadata=<metadata>"meta":[{"place":{"plac
>>>
>>> e_type":"city","region":"CA+"}},{"song":{"artist":"Prince","songtitle":"Never+L
>>> et+Me+Down"}}]</metadata>'
>>> http://localhost:8080/esme-server-apache-esme-1.0-RC
>>> 1-incubating/api2/user/messages
>>> <?xml version="1.0" encoding="UTF-8"?>
>>> <api><message>
>>>  <id>10</id>
>>>  <date>Sat, 17 Jul 2010 15:58:44 UTC</date>
>>>  <source>api2</source>
>>>  <body>test72</body>
>>>
>>>
>>> <metadata>&quot;meta&quot;:[{&quot;place&quot;:{&quot;place_type&quot;:&quot;c
>>> ity&quot;,&quot;region&quot;:&quot;CA
>>> &quot;}},{&quot;song&quot;:{&quot;artist&q
>>> uot;:&quot;Prince&quot;,&quot;songtitle&quot;:&quot;Never
Let Me
>>> Down&quot;}}]</
>>> metadata>
>>>  <author><nickname>imtiaz2</nickname><id>3</id></author>
>>>  <tags></tags><replyto></replyto><conversation></conversation>
>>> </message></api>
>>>
>>> imtiaz@imtiaz-20100131 /cygdrive/d/temp
>>> $
>>>
>>> Imtiaz
>>>
>>> ----- Original Message ----- From: "Ethan Jewett" <esjewett@gmail.com>
>>> To: <esme-dev@incubator.apache.org>; <in.imtiaz@gmail.com>
>>> Sent: Saturday, July 17, 2010 8:30 PM
>>> Subject: Re: Release planning
>>>
>>>
>>> Re ESME-242 - I think you (and Vassil) are right, but I haven't been
>>> able to independently confirm and I haven't seen a working API call
>>> attached to the Jira issue, so I'm not willing to close it yet. If we
>>> had working metadata examples using properly escaped XML and JSON, I
>>> would be very happy :-)
>>>
>>> Ethan
>>>
>>> On Sat, Jul 17, 2010 at 4:32 PM,  <in.imtiaz@gmail.com> wrote:
>>>>
>>>> About ESME-242,
>>>> Per Vassil Dichev's mail to this group that http POST text used in the
>>>> shell curl command must not be quoted and must be url encoded or must be
>>>> quoted using single quotes, this Jira issue should be resolved to 'not a
>>>> bug', if I recall correctly, that is!
>>>> Imtiaz
>>>>
>>>> Sent from BlackBerryŽ on Airtel
>>>>
>>>> -----Original Message-----
>>>> From: Ethan Jewett <esjewett@gmail.com>
>>>> Date: Sat, 17 Jul 2010 16:11:44
>>>> To: <esme-dev@incubator.apache.org>
>>>> Reply-To: esme-dev@incubator.apache.org
>>>> Subject: Re: Release planning
>>>>
>>>> Thoughts on items remaining for this release:
>>>>
>>>> ESME-225 - "need unit test for GET api2/users/actions don't retrieve
>>>> disabled actions" - I don't think this is required for release and
>>>> should be moved to the backlog.
>>>>
>>>> ESME-242 - "The use of metadata in the api2 currently doesn't work" -
>>>> I think this is important but also not required for release and should
>>>> be moved to the backlog.
>>>>
>>>> I agree that ESME-205 and ESME-212 are major. ESME-212 at least is a
>>>> blocker.
>>>>
>>>>
>>>>
>>>> Regarding changes to be made to API2: I've created JIRA issues for all
>>>> the todos at the bottom of the file (except a couple that were already
>>>> done). It was lazy of me to put a todo-list in the file - I should
>>>> really avoid that and force myself to have everything in Jira. I will
>>>> be checking in a version of the file without the todo-list shortly.
>>>>
>>>>
>>>> Cheers,
>>>> Ethan
>>>>
>>>> On Fri, Jul 16, 2010 at 12:40 PM, Richard Hirsch <hirsch.dick@gmail.com>
>>>> wrote:
>>>>>
>>>>> I've been working on the JIRA items and we have 10 items left for this
>>>>> release.
>>>>>
>>>>> The following items are linked to IE 7:
>>>>> * ESME-207
>>>>> * ESME-239
>>>>> * ESME-208
>>>>>
>>>>> Some of these items are CSS-related (for example, 207) but the other
>>>>> ones are javascript / jquery related.
>>>>>
>>>>> I think the two main issues are:
>>>>>
>>>>> * ESME-205 Search is broken (This only happens when you use a JDBC
>>>>> based compass indexes - first noticed on Stax but it also is present
>>>>> locally)
>>>>> * ESME-212 Messages from pools aren't hidden
>>>>>
>>>>> I'm going to be gone the next four weeks on vacation, so either
>>>>> someone else can coordinate the release or we wait until I return to
>>>>> continue with our release preparations.
>>>>>
>>>>> If anyone wants to work on other items until the release, here are
>>>>> other items that I consider important:
>>>>>
>>>>> * ESME-108 - View my pools" functionality
>>>>> * ESME-170 Pubsubhubbub support for Atom & RSS actions#
>>>>> * ESME-214 Add container-based authentication
>>>>> * ESME-213 Twitter API is out-of-date and API calls don't always
>>>>> return the correct info
>>>>>
>>>>> -- other api2 related changes - see the ToDos at the bottom of the
>>>>> API2.scala page .
>>>>>
>>>>> D.
>>>>>
>>>>
>>>
>>
>>
>
Mime
View raw message