incubator-esme-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ethan Jewett <esjew...@gmail.com>
Subject Re: Metadata handling (was "Release planning")
Date Sun, 18 Jul 2010 08:01:30 GMT
I've updated the Jira ESME-242 with examples of how it does work and
how it should work, as well as a suggestion that it is Message.scala
that needs to change, rather than the API code.

https://issues.apache.org/jira/browse/ESME-242?focusedCommentId=12889585&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12889585

Ethan

On Sun, Jul 18, 2010 at 8:36 AM, Richard Hirsch <hirsch.dick@gmail.com> wrote:
> On Sun, Jul 18, 2010 at 3:58 AM, Imtiaz Ahmed H E <in.imtiaz@gmail.com> wrote:
>> Ethan, you said,
>>
>> <<
>> We could just put code into the API to make this behave how we want by
>> URLencoding it wrapping it in a special XML tag before storing it as
>> part of the message, then un-entity encode it and then un-URLencode it
>> before sending it back to the client, but that is going to result in a
>> horrible mess stored in the actual message. I think it will be better
>> to change the way that the actual Message class handles metadata.
>>>>
>>
>> Sounds okay...
>>
>> If you can specify in the Jira issue itself, how this should be handled I
>> can take that as decided and go ahead and implement it.
>> And, does Dick (Hirsch) need to okay it before you do that? I don't know how
>> it works on Esme...
>
> Although Anne and I attempt to coordinate things, Apache is a
> community effort. The community decides what is to be done. Anyone can
> create JIRA items and work on them.
>
> @Imtiaz - During my vacation, you'll have to ping this list when you
> attach patches to Jira items. Someone else will have to commit the
> changes to SVN.
>
>>
>> Imtiaz
>>
>> ----- Original Message ----- From: "Ethan Jewett" <esjewett@gmail.com>
>> To: "Imtiaz Ahmed H E" <in.imtiaz@gmail.com>
>> Cc: <esme-dev@incubator.apache.org>
>> Sent: Saturday, July 17, 2010 11:38 PM
>> Subject: Metadata handling (was "Release planning")
>>
>>
>> Oh .... I see how it works. Thanks for the example. I was never able
>> to understand this before. This functionality is not what we want, but
>> just for completeness, here is a summary of how passing in metadata
>> through the API currently works (basic XML example):
>>
>> Send in:
>> <outer><meta><metameta>Hello</metameta></meta><onlymeta>Meta</onlymeta></outer>
>>
>> Get out: HelloMeta
>>
>> Note: this is the text contents of the XML tags concatenated together
>> in "order", whatever "order" means for XML content.
>>
>> Here's a JSON example:
>>
>> Send in:
>> <anytag>"meta":[{"place":{"place_type":"city","region":"CA+"}},{"song":{"artist":"Prince","songtitle":"Never+Let+Me+Down"}}]</anytag>
>>
>> Get out:
>> &quot;meta&quot;:[{&quot;place&quot;:{&quot;place_type&quot;:&quot;city&quot;,&quot;region&quot;:&quot;CA+&quot;}},{&quot;song&quot;:{&quot;artist&quot;:&quot;Prince&quot;,&quot;songtitle&quot;:&quot;Never+Let+Me+Down&quot;}}]
>>
>> Note: This is, again, the text contents of the XML tags. The output is
>> HTML entity encoded, as it was before, but there weren't any
>> characters in the previous example that would have been encoded.
>>
>> Second note: The input *must* be wrapped in valid XML tags or it is ignored.
>>
>> I believe that this is completely wrong. The API should simply take
>> the text assigned to the metadata parameter, store it as part of the
>> message, and return exactly the same string in the message response.
>> It should not care what is in the text and it should not modify it in
>> any way.
>>
>> I think that making this work is going to take some revamping of the
>> internal handling of metadata, moving any kind of parsing and/or HTML
>> entity encoding to the UI code and have the internal Message model
>> simply store the metadata without thinking about it so hard. This
>> could clearly result in breakage, so I'm not sure how best to proceed.
>>
>> We could just put code into the API to make this behave how we want by
>> URLencoding it wrapping it in a special XML tag before storing it as
>> part of the message, then un-entity encode it and then un-URLencode it
>> before sending it back to the client, but that is going to result in a
>> horrible mess stored in the actual message. I think it will be better
>> to change the way that the actual Message class handles metadata.
>>
>> Does anyone know what functionality of the web UI currently uses
>> metadata, so that we can test for breakage if we change how this
>> works?
>>
>> Imtiaz, if you want to work on making this behave in a sane way, that
>> would be fantastic. It's a big job, but we've got to start somewhere.
>> I think ESME-242 is a good umbrella issue for this and we can modify
>> the issue to better reflect the problem.
>>
>> Ethan
>>
>> On Sat, Jul 17, 2010 at 6:57 PM, Imtiaz Ahmed H E <in.imtiaz@gmail.com>
>> wrote:
>>>
>>> For example, for the response of the following (GET) maybe we need to url
>>> decode the response before we (the api) respond with it ? (Reference: see
>>> the reponse to the POST of a message in my previous mail, forwarded
>>> below).
>>>
>>> curl -b headers
>>>
>>> http://localhost:8080/esme-server-apache-esme-1.0-RC1-incubating/api2/user/messages
>>>
>>> Of course this may be a typical naive one from me...! But let me know.
>>> Dick
>>> had said maybe I look at 242 before Vassil made his point & this
>>> discussion
>>> took place...
>>>
>>> Imtiaz
>>>
>>> ----- Original Message ----- From: "Imtiaz Ahmed H E"
>>> <in.imtiaz@gmail.com>
>>> To: <esme-dev@incubator.apache.org>; "Ethan Jewett" <esjewett@gmail.com>
>>> Sent: Saturday, July 17, 2010 9:35 PM
>>> Subject: Re: Release planning
>>>
>>>
>>>> Well...here you are...
>>>>
>>>> imtiaz@imtiaz-20100131 /cygdrive/d/temp
>>>> $ curl --dump-header headers -d "token=HEZTQKM525SAMIPN4EDVRUOGHI40AKBL"
>>>> http:/
>>>> /localhost:8080/esme-server-apache-esme-1.0-RC1-incubating/api2/session
>>>> <?xml version="1.0" encoding="UTF-8"?>
>>>>
>>>>
>>>> <api><session><user><id>3</id><nickname>imtiaz2</nickname><image>None</image><wh
>>>> ole_name>I A 2 H E</whole_name></user></session></api>
>>>>
>>>> imtiaz@imtiaz-20100131 /cygdrive/d/temp
>>>> $ cat headers
>>>> HTTP/1.1 200 OK
>>>> Server: Apache-Coyote/1.1
>>>> Set-Cookie: JSESSIONID=28DBA5607A4D94C9E6E2BD1E61492EAD;
>>>> Path=/esme-server-apach
>>>> e-esme-1.0-RC1-incubating
>>>> Expires: Sat, 17 Jul 2010 15:58:22 UTC
>>>> Date: Sat, 17 Jul 2010 15:58:22 GMT
>>>> Pragma: no-cache
>>>> Cache-Control: no-cache; private; no-store
>>>> X-Lift-Version: 2.0-SNAPSHOT
>>>> Content-Type: text/xml;charset=utf-8
>>>> Content-Length: 178
>>>>
>>>>
>>>> imtiaz@imtiaz-20100131 /cygdrive/d/temp
>>>> $ curl -b headers -d
>>>> 'message=test72&metadata=<metadata>"meta":[{"place":{"plac
>>>>
>>>>
>>>> e_type":"city","region":"CA+"}},{"song":{"artist":"Prince","songtitle":"Never+L
>>>> et+Me+Down"}}]</metadata>'
>>>> http://localhost:8080/esme-server-apache-esme-1.0-RC
>>>> 1-incubating/api2/user/messages
>>>> <?xml version="1.0" encoding="UTF-8"?>
>>>> <api><message>
>>>> <id>10</id>
>>>> <date>Sat, 17 Jul 2010 15:58:44 UTC</date>
>>>> <source>api2</source>
>>>> <body>test72</body>
>>>>
>>>>
>>>>
>>>> <metadata>&quot;meta&quot;:[{&quot;place&quot;:{&quot;place_type&quot;:&quot;c
>>>> ity&quot;,&quot;region&quot;:&quot;CA
>>>> &quot;}},{&quot;song&quot;:{&quot;artist&q
>>>> uot;:&quot;Prince&quot;,&quot;songtitle&quot;:&quot;Never
Let Me
>>>> Down&quot;}}]</
>>>> metadata>
>>>> <author><nickname>imtiaz2</nickname><id>3</id></author>
>>>> <tags></tags><replyto></replyto><conversation></conversation>
>>>> </message></api>
>>>>
>>>> imtiaz@imtiaz-20100131 /cygdrive/d/temp
>>>> $
>>>>
>>>> Imtiaz
>>>>
>>>> ----- Original Message ----- From: "Ethan Jewett" <esjewett@gmail.com>
>>>> To: <esme-dev@incubator.apache.org>; <in.imtiaz@gmail.com>
>>>> Sent: Saturday, July 17, 2010 8:30 PM
>>>> Subject: Re: Release planning
>>>>
>>>>
>>>> Re ESME-242 - I think you (and Vassil) are right, but I haven't been
>>>> able to independently confirm and I haven't seen a working API call
>>>> attached to the Jira issue, so I'm not willing to close it yet. If we
>>>> had working metadata examples using properly escaped XML and JSON, I
>>>> would be very happy :-)
>>>>
>>>> Ethan
>>>>
>>>> On Sat, Jul 17, 2010 at 4:32 PM, <in.imtiaz@gmail.com> wrote:
>>>>>
>>>>> About ESME-242,
>>>>> Per Vassil Dichev's mail to this group that http POST text used in the
>>>>> shell curl command must not be quoted and must be url encoded or must
be
>>>>> quoted using single quotes, this Jira issue should be resolved to 'not
a
>>>>> bug', if I recall correctly, that is!
>>>>> Imtiaz
>>>>>
>>>>> Sent from BlackBerryĹ˝ on Airtel
>>>>>
>>>>> -----Original Message-----
>>>>> From: Ethan Jewett <esjewett@gmail.com>
>>>>> Date: Sat, 17 Jul 2010 16:11:44
>>>>> To: <esme-dev@incubator.apache.org>
>>>>> Reply-To: esme-dev@incubator.apache.org
>>>>> Subject: Re: Release planning
>>>>>
>>>>> Thoughts on items remaining for this release:
>>>>>
>>>>> ESME-225 - "need unit test for GET api2/users/actions don't retrieve
>>>>> disabled actions" - I don't think this is required for release and
>>>>> should be moved to the backlog.
>>>>>
>>>>> ESME-242 - "The use of metadata in the api2 currently doesn't work" -
>>>>> I think this is important but also not required for release and should
>>>>> be moved to the backlog.
>>>>>
>>>>> I agree that ESME-205 and ESME-212 are major. ESME-212 at least is a
>>>>> blocker.
>>>>>
>>>>>
>>>>>
>>>>> Regarding changes to be made to API2: I've created JIRA issues for all
>>>>> the todos at the bottom of the file (except a couple that were already
>>>>> done). It was lazy of me to put a todo-list in the file - I should
>>>>> really avoid that and force myself to have everything in Jira. I will
>>>>> be checking in a version of the file without the todo-list shortly.
>>>>>
>>>>>
>>>>> Cheers,
>>>>> Ethan
>>>>>
>>>>> On Fri, Jul 16, 2010 at 12:40 PM, Richard Hirsch <hirsch.dick@gmail.com>
>>>>> wrote:
>>>>>>
>>>>>> I've been working on the JIRA items and we have 10 items left for
this
>>>>>> release.
>>>>>>
>>>>>> The following items are linked to IE 7:
>>>>>> * ESME-207
>>>>>> * ESME-239
>>>>>> * ESME-208
>>>>>>
>>>>>> Some of these items are CSS-related (for example, 207) but the other
>>>>>> ones are javascript / jquery related.
>>>>>>
>>>>>> I think the two main issues are:
>>>>>>
>>>>>> * ESME-205 Search is broken (This only happens when you use a JDBC
>>>>>> based compass indexes - first noticed on Stax but it also is present
>>>>>> locally)
>>>>>> * ESME-212 Messages from pools aren't hidden
>>>>>>
>>>>>> I'm going to be gone the next four weeks on vacation, so either
>>>>>> someone else can coordinate the release or we wait until I return
to
>>>>>> continue with our release preparations.
>>>>>>
>>>>>> If anyone wants to work on other items until the release, here are
>>>>>> other items that I consider important:
>>>>>>
>>>>>> * ESME-108 - View my pools" functionality
>>>>>> * ESME-170 Pubsubhubbub support for Atom & RSS actions#
>>>>>> * ESME-214 Add container-based authentication
>>>>>> * ESME-213 Twitter API is out-of-date and API calls don't always
>>>>>> return the correct info
>>>>>>
>>>>>> -- other api2 related changes - see the ToDos at the bottom of the
>>>>>> API2.scala page .
>>>>>>
>>>>>> D.
>>>>>>
>>>>>
>>>>
>>>
>>>
>>
>>
>

Mime
View raw message