incubator-esme-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Richard Hirsch <hirsch.d...@gmail.com>
Subject Re: Metadata handling (was "Release planning")
Date Sun, 18 Jul 2010 06:36:12 GMT
On Sun, Jul 18, 2010 at 3:58 AM, Imtiaz Ahmed H E <in.imtiaz@gmail.com> wrote:
> Ethan, you said,
>
> <<
> We could just put code into the API to make this behave how we want by
> URLencoding it wrapping it in a special XML tag before storing it as
> part of the message, then un-entity encode it and then un-URLencode it
> before sending it back to the client, but that is going to result in a
> horrible mess stored in the actual message. I think it will be better
> to change the way that the actual Message class handles metadata.
>>>
>
> Sounds okay...
>
> If you can specify in the Jira issue itself, how this should be handled I
> can take that as decided and go ahead and implement it.
> And, does Dick (Hirsch) need to okay it before you do that? I don't know how
> it works on Esme...

Although Anne and I attempt to coordinate things, Apache is a
community effort. The community decides what is to be done. Anyone can
create JIRA items and work on them.

@Imtiaz - During my vacation, you'll have to ping this list when you
attach patches to Jira items. Someone else will have to commit the
changes to SVN.

>
> Imtiaz
>
> ----- Original Message ----- From: "Ethan Jewett" <esjewett@gmail.com>
> To: "Imtiaz Ahmed H E" <in.imtiaz@gmail.com>
> Cc: <esme-dev@incubator.apache.org>
> Sent: Saturday, July 17, 2010 11:38 PM
> Subject: Metadata handling (was "Release planning")
>
>
> Oh .... I see how it works. Thanks for the example. I was never able
> to understand this before. This functionality is not what we want, but
> just for completeness, here is a summary of how passing in metadata
> through the API currently works (basic XML example):
>
> Send in:
> <outer><meta><metameta>Hello</metameta></meta><onlymeta>Meta</onlymeta></outer>
>
> Get out: HelloMeta
>
> Note: this is the text contents of the XML tags concatenated together
> in "order", whatever "order" means for XML content.
>
> Here's a JSON example:
>
> Send in:
> <anytag>"meta":[{"place":{"place_type":"city","region":"CA+"}},{"song":{"artist":"Prince","songtitle":"Never+Let+Me+Down"}}]</anytag>
>
> Get out:
> &quot;meta&quot;:[{&quot;place&quot;:{&quot;place_type&quot;:&quot;city&quot;,&quot;region&quot;:&quot;CA+&quot;}},{&quot;song&quot;:{&quot;artist&quot;:&quot;Prince&quot;,&quot;songtitle&quot;:&quot;Never+Let+Me+Down&quot;}}]
>
> Note: This is, again, the text contents of the XML tags. The output is
> HTML entity encoded, as it was before, but there weren't any
> characters in the previous example that would have been encoded.
>
> Second note: The input *must* be wrapped in valid XML tags or it is ignored.
>
> I believe that this is completely wrong. The API should simply take
> the text assigned to the metadata parameter, store it as part of the
> message, and return exactly the same string in the message response.
> It should not care what is in the text and it should not modify it in
> any way.
>
> I think that making this work is going to take some revamping of the
> internal handling of metadata, moving any kind of parsing and/or HTML
> entity encoding to the UI code and have the internal Message model
> simply store the metadata without thinking about it so hard. This
> could clearly result in breakage, so I'm not sure how best to proceed.
>
> We could just put code into the API to make this behave how we want by
> URLencoding it wrapping it in a special XML tag before storing it as
> part of the message, then un-entity encode it and then un-URLencode it
> before sending it back to the client, but that is going to result in a
> horrible mess stored in the actual message. I think it will be better
> to change the way that the actual Message class handles metadata.
>
> Does anyone know what functionality of the web UI currently uses
> metadata, so that we can test for breakage if we change how this
> works?
>
> Imtiaz, if you want to work on making this behave in a sane way, that
> would be fantastic. It's a big job, but we've got to start somewhere.
> I think ESME-242 is a good umbrella issue for this and we can modify
> the issue to better reflect the problem.
>
> Ethan
>
> On Sat, Jul 17, 2010 at 6:57 PM, Imtiaz Ahmed H E <in.imtiaz@gmail.com>
> wrote:
>>
>> For example, for the response of the following (GET) maybe we need to url
>> decode the response before we (the api) respond with it ? (Reference: see
>> the reponse to the POST of a message in my previous mail, forwarded
>> below).
>>
>> curl -b headers
>>
>> http://localhost:8080/esme-server-apache-esme-1.0-RC1-incubating/api2/user/messages
>>
>> Of course this may be a typical naive one from me...! But let me know.
>> Dick
>> had said maybe I look at 242 before Vassil made his point & this
>> discussion
>> took place...
>>
>> Imtiaz
>>
>> ----- Original Message ----- From: "Imtiaz Ahmed H E"
>> <in.imtiaz@gmail.com>
>> To: <esme-dev@incubator.apache.org>; "Ethan Jewett" <esjewett@gmail.com>
>> Sent: Saturday, July 17, 2010 9:35 PM
>> Subject: Re: Release planning
>>
>>
>>> Well...here you are...
>>>
>>> imtiaz@imtiaz-20100131 /cygdrive/d/temp
>>> $ curl --dump-header headers -d "token=HEZTQKM525SAMIPN4EDVRUOGHI40AKBL"
>>> http:/
>>> /localhost:8080/esme-server-apache-esme-1.0-RC1-incubating/api2/session
>>> <?xml version="1.0" encoding="UTF-8"?>
>>>
>>>
>>> <api><session><user><id>3</id><nickname>imtiaz2</nickname><image>None</image><wh
>>> ole_name>I A 2 H E</whole_name></user></session></api>
>>>
>>> imtiaz@imtiaz-20100131 /cygdrive/d/temp
>>> $ cat headers
>>> HTTP/1.1 200 OK
>>> Server: Apache-Coyote/1.1
>>> Set-Cookie: JSESSIONID=28DBA5607A4D94C9E6E2BD1E61492EAD;
>>> Path=/esme-server-apach
>>> e-esme-1.0-RC1-incubating
>>> Expires: Sat, 17 Jul 2010 15:58:22 UTC
>>> Date: Sat, 17 Jul 2010 15:58:22 GMT
>>> Pragma: no-cache
>>> Cache-Control: no-cache; private; no-store
>>> X-Lift-Version: 2.0-SNAPSHOT
>>> Content-Type: text/xml;charset=utf-8
>>> Content-Length: 178
>>>
>>>
>>> imtiaz@imtiaz-20100131 /cygdrive/d/temp
>>> $ curl -b headers -d
>>> 'message=test72&metadata=<metadata>"meta":[{"place":{"plac
>>>
>>>
>>> e_type":"city","region":"CA+"}},{"song":{"artist":"Prince","songtitle":"Never+L
>>> et+Me+Down"}}]</metadata>'
>>> http://localhost:8080/esme-server-apache-esme-1.0-RC
>>> 1-incubating/api2/user/messages
>>> <?xml version="1.0" encoding="UTF-8"?>
>>> <api><message>
>>> <id>10</id>
>>> <date>Sat, 17 Jul 2010 15:58:44 UTC</date>
>>> <source>api2</source>
>>> <body>test72</body>
>>>
>>>
>>>
>>> <metadata>&quot;meta&quot;:[{&quot;place&quot;:{&quot;place_type&quot;:&quot;c
>>> ity&quot;,&quot;region&quot;:&quot;CA
>>> &quot;}},{&quot;song&quot;:{&quot;artist&q
>>> uot;:&quot;Prince&quot;,&quot;songtitle&quot;:&quot;Never
Let Me
>>> Down&quot;}}]</
>>> metadata>
>>> <author><nickname>imtiaz2</nickname><id>3</id></author>
>>> <tags></tags><replyto></replyto><conversation></conversation>
>>> </message></api>
>>>
>>> imtiaz@imtiaz-20100131 /cygdrive/d/temp
>>> $
>>>
>>> Imtiaz
>>>
>>> ----- Original Message ----- From: "Ethan Jewett" <esjewett@gmail.com>
>>> To: <esme-dev@incubator.apache.org>; <in.imtiaz@gmail.com>
>>> Sent: Saturday, July 17, 2010 8:30 PM
>>> Subject: Re: Release planning
>>>
>>>
>>> Re ESME-242 - I think you (and Vassil) are right, but I haven't been
>>> able to independently confirm and I haven't seen a working API call
>>> attached to the Jira issue, so I'm not willing to close it yet. If we
>>> had working metadata examples using properly escaped XML and JSON, I
>>> would be very happy :-)
>>>
>>> Ethan
>>>
>>> On Sat, Jul 17, 2010 at 4:32 PM, <in.imtiaz@gmail.com> wrote:
>>>>
>>>> About ESME-242,
>>>> Per Vassil Dichev's mail to this group that http POST text used in the
>>>> shell curl command must not be quoted and must be url encoded or must be
>>>> quoted using single quotes, this Jira issue should be resolved to 'not a
>>>> bug', if I recall correctly, that is!
>>>> Imtiaz
>>>>
>>>> Sent from BlackBerryĹ˝ on Airtel
>>>>
>>>> -----Original Message-----
>>>> From: Ethan Jewett <esjewett@gmail.com>
>>>> Date: Sat, 17 Jul 2010 16:11:44
>>>> To: <esme-dev@incubator.apache.org>
>>>> Reply-To: esme-dev@incubator.apache.org
>>>> Subject: Re: Release planning
>>>>
>>>> Thoughts on items remaining for this release:
>>>>
>>>> ESME-225 - "need unit test for GET api2/users/actions don't retrieve
>>>> disabled actions" - I don't think this is required for release and
>>>> should be moved to the backlog.
>>>>
>>>> ESME-242 - "The use of metadata in the api2 currently doesn't work" -
>>>> I think this is important but also not required for release and should
>>>> be moved to the backlog.
>>>>
>>>> I agree that ESME-205 and ESME-212 are major. ESME-212 at least is a
>>>> blocker.
>>>>
>>>>
>>>>
>>>> Regarding changes to be made to API2: I've created JIRA issues for all
>>>> the todos at the bottom of the file (except a couple that were already
>>>> done). It was lazy of me to put a todo-list in the file - I should
>>>> really avoid that and force myself to have everything in Jira. I will
>>>> be checking in a version of the file without the todo-list shortly.
>>>>
>>>>
>>>> Cheers,
>>>> Ethan
>>>>
>>>> On Fri, Jul 16, 2010 at 12:40 PM, Richard Hirsch <hirsch.dick@gmail.com>
>>>> wrote:
>>>>>
>>>>> I've been working on the JIRA items and we have 10 items left for this
>>>>> release.
>>>>>
>>>>> The following items are linked to IE 7:
>>>>> * ESME-207
>>>>> * ESME-239
>>>>> * ESME-208
>>>>>
>>>>> Some of these items are CSS-related (for example, 207) but the other
>>>>> ones are javascript / jquery related.
>>>>>
>>>>> I think the two main issues are:
>>>>>
>>>>> * ESME-205 Search is broken (This only happens when you use a JDBC
>>>>> based compass indexes - first noticed on Stax but it also is present
>>>>> locally)
>>>>> * ESME-212 Messages from pools aren't hidden
>>>>>
>>>>> I'm going to be gone the next four weeks on vacation, so either
>>>>> someone else can coordinate the release or we wait until I return to
>>>>> continue with our release preparations.
>>>>>
>>>>> If anyone wants to work on other items until the release, here are
>>>>> other items that I consider important:
>>>>>
>>>>> * ESME-108 - View my pools" functionality
>>>>> * ESME-170 Pubsubhubbub support for Atom & RSS actions#
>>>>> * ESME-214 Add container-based authentication
>>>>> * ESME-213 Twitter API is out-of-date and API calls don't always
>>>>> return the correct info
>>>>>
>>>>> -- other api2 related changes - see the ToDos at the bottom of the
>>>>> API2.scala page .
>>>>>
>>>>> D.
>>>>>
>>>>
>>>
>>
>>
>
>

Mime
View raw message