incubator-esme-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ethan Jewett <esjew...@gmail.com>
Subject Re: Metadata handling (was "Release planning")
Date Sun, 18 Jul 2010 08:14:15 GMT
Done!

On Sun, Jul 18, 2010 at 10:09 AM, Imtiaz Ahmed H E <in.imtiaz@gmail.com> wrote:
> Thanks.
>
> Hope Vassil can take a look at it and maybe whet it if required...
>
> Will try and get going on it...Ethan would you assign Jira ESME-242 to me,
> since I don't feel I should start assigning Jira items to myself at this
> stage when I'm a little unaware of many aspects of Esme.
>
> Thanks
> Imtiaz
>
> ----- Original Message ----- From: "Ethan Jewett" <esjewett@gmail.com>
> To: <esme-dev@incubator.apache.org>
> Sent: Sunday, July 18, 2010 1:31 PM
> Subject: Re: Metadata handling (was "Release planning")
>
>
> I've updated the Jira ESME-242 with examples of how it does work and
> how it should work, as well as a suggestion that it is Message.scala
> that needs to change, rather than the API code.
>
> https://issues.apache.org/jira/browse/ESME-242?focusedCommentId=12889585&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12889585
>
> Ethan
>
> On Sun, Jul 18, 2010 at 8:36 AM, Richard Hirsch <hirsch.dick@gmail.com>
> wrote:
>>
>> On Sun, Jul 18, 2010 at 3:58 AM, Imtiaz Ahmed H E <in.imtiaz@gmail.com>
>> wrote:
>>>
>>> Ethan, you said,
>>>
>>> <<
>>> We could just put code into the API to make this behave how we want by
>>> URLencoding it wrapping it in a special XML tag before storing it as
>>> part of the message, then un-entity encode it and then un-URLencode it
>>> before sending it back to the client, but that is going to result in a
>>> horrible mess stored in the actual message. I think it will be better
>>> to change the way that the actual Message class handles metadata.
>>>>>
>>>
>>> Sounds okay...
>>>
>>> If you can specify in the Jira issue itself, how this should be handled I
>>> can take that as decided and go ahead and implement it.
>>> And, does Dick (Hirsch) need to okay it before you do that? I don't know
>>> how
>>> it works on Esme...
>>
>> Although Anne and I attempt to coordinate things, Apache is a
>> community effort. The community decides what is to be done. Anyone can
>> create JIRA items and work on them.
>>
>> @Imtiaz - During my vacation, you'll have to ping this list when you
>> attach patches to Jira items. Someone else will have to commit the
>> changes to SVN.
>>
>>>
>>> Imtiaz
>>>
>>> ----- Original Message ----- From: "Ethan Jewett" <esjewett@gmail.com>
>>> To: "Imtiaz Ahmed H E" <in.imtiaz@gmail.com>
>>> Cc: <esme-dev@incubator.apache.org>
>>> Sent: Saturday, July 17, 2010 11:38 PM
>>> Subject: Metadata handling (was "Release planning")
>>>
>>>
>>> Oh .... I see how it works. Thanks for the example. I was never able
>>> to understand this before. This functionality is not what we want, but
>>> just for completeness, here is a summary of how passing in metadata
>>> through the API currently works (basic XML example):
>>>
>>> Send in:
>>>
>>> <outer><meta><metameta>Hello</metameta></meta><onlymeta>Meta</onlymeta></outer>
>>>
>>> Get out: HelloMeta
>>>
>>> Note: this is the text contents of the XML tags concatenated together
>>> in "order", whatever "order" means for XML content.
>>>
>>> Here's a JSON example:
>>>
>>> Send in:
>>>
>>> <anytag>"meta":[{"place":{"place_type":"city","region":"CA+"}},{"song":{"artist":"Prince","songtitle":"Never+Let+Me+Down"}}]</anytag>
>>>
>>> Get out:
>>>
>>> &quot;meta&quot;:[{&quot;place&quot;:{&quot;place_type&quot;:&quot;city&quot;,&quot;region&quot;:&quot;CA+&quot;}},{&quot;song&quot;:{&quot;artist&quot;:&quot;Prince&quot;,&quot;songtitle&quot;:&quot;Never+Let+Me+Down&quot;}}]
>>>
>>> Note: This is, again, the text contents of the XML tags. The output is
>>> HTML entity encoded, as it was before, but there weren't any
>>> characters in the previous example that would have been encoded.
>>>
>>> Second note: The input *must* be wrapped in valid XML tags or it is
>>> ignored.
>>>
>>> I believe that this is completely wrong. The API should simply take
>>> the text assigned to the metadata parameter, store it as part of the
>>> message, and return exactly the same string in the message response.
>>> It should not care what is in the text and it should not modify it in
>>> any way.
>>>
>>> I think that making this work is going to take some revamping of the
>>> internal handling of metadata, moving any kind of parsing and/or HTML
>>> entity encoding to the UI code and have the internal Message model
>>> simply store the metadata without thinking about it so hard. This
>>> could clearly result in breakage, so I'm not sure how best to proceed.
>>>
>>> We could just put code into the API to make this behave how we want by
>>> URLencoding it wrapping it in a special XML tag before storing it as
>>> part of the message, then un-entity encode it and then un-URLencode it
>>> before sending it back to the client, but that is going to result in a
>>> horrible mess stored in the actual message. I think it will be better
>>> to change the way that the actual Message class handles metadata.
>>>
>>> Does anyone know what functionality of the web UI currently uses
>>> metadata, so that we can test for breakage if we change how this
>>> works?
>>>
>>> Imtiaz, if you want to work on making this behave in a sane way, that
>>> would be fantastic. It's a big job, but we've got to start somewhere.
>>> I think ESME-242 is a good umbrella issue for this and we can modify
>>> the issue to better reflect the problem.
>>>
>>> Ethan
>>>
>>> On Sat, Jul 17, 2010 at 6:57 PM, Imtiaz Ahmed H E <in.imtiaz@gmail.com>
>>> wrote:
>>>>
>>>> For example, for the response of the following (GET) maybe we need to
>>>> url
>>>> decode the response before we (the api) respond with it ? (Reference:
>>>> see
>>>> the reponse to the POST of a message in my previous mail, forwarded
>>>> below).
>>>>
>>>> curl -b headers
>>>>
>>>>
>>>> http://localhost:8080/esme-server-apache-esme-1.0-RC1-incubating/api2/user/messages
>>>>
>>>> Of course this may be a typical naive one from me...! But let me know.
>>>> Dick
>>>> had said maybe I look at 242 before Vassil made his point & this
>>>> discussion
>>>> took place...
>>>>
>>>> Imtiaz
>>>>
>>>> ----- Original Message ----- From: "Imtiaz Ahmed H E"
>>>> <in.imtiaz@gmail.com>
>>>> To: <esme-dev@incubator.apache.org>; "Ethan Jewett" <esjewett@gmail.com>
>>>> Sent: Saturday, July 17, 2010 9:35 PM
>>>> Subject: Re: Release planning
>>>>
>>>>
>>>>> Well...here you are...
>>>>>
>>>>> imtiaz@imtiaz-20100131 /cygdrive/d/temp
>>>>> $ curl --dump-header headers -d
>>>>> "token=HEZTQKM525SAMIPN4EDVRUOGHI40AKBL"
>>>>> http:/
>>>>> /localhost:8080/esme-server-apache-esme-1.0-RC1-incubating/api2/session
>>>>> <?xml version="1.0" encoding="UTF-8"?>
>>>>>
>>>>>
>>>>>
>>>>> <api><session><user><id>3</id><nickname>imtiaz2</nickname><image>None</image><wh
>>>>> ole_name>I A 2 H E</whole_name></user></session></api>
>>>>>
>>>>> imtiaz@imtiaz-20100131 /cygdrive/d/temp
>>>>> $ cat headers
>>>>> HTTP/1.1 200 OK
>>>>> Server: Apache-Coyote/1.1
>>>>> Set-Cookie: JSESSIONID=28DBA5607A4D94C9E6E2BD1E61492EAD;
>>>>> Path=/esme-server-apach
>>>>> e-esme-1.0-RC1-incubating
>>>>> Expires: Sat, 17 Jul 2010 15:58:22 UTC
>>>>> Date: Sat, 17 Jul 2010 15:58:22 GMT
>>>>> Pragma: no-cache
>>>>> Cache-Control: no-cache; private; no-store
>>>>> X-Lift-Version: 2.0-SNAPSHOT
>>>>> Content-Type: text/xml;charset=utf-8
>>>>> Content-Length: 178
>>>>>
>>>>>
>>>>> imtiaz@imtiaz-20100131 /cygdrive/d/temp
>>>>> $ curl -b headers -d
>>>>> 'message=test72&metadata=<metadata>"meta":[{"place":{"plac
>>>>>
>>>>>
>>>>>
>>>>> e_type":"city","region":"CA+"}},{"song":{"artist":"Prince","songtitle":"Never+L
>>>>> et+Me+Down"}}]</metadata>'
>>>>> http://localhost:8080/esme-server-apache-esme-1.0-RC
>>>>> 1-incubating/api2/user/messages
>>>>> <?xml version="1.0" encoding="UTF-8"?>
>>>>> <api><message>
>>>>> <id>10</id>
>>>>> <date>Sat, 17 Jul 2010 15:58:44 UTC</date>
>>>>> <source>api2</source>
>>>>> <body>test72</body>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> <metadata>&quot;meta&quot;:[{&quot;place&quot;:{&quot;place_type&quot;:&quot;c
>>>>> ity&quot;,&quot;region&quot;:&quot;CA
>>>>> &quot;}},{&quot;song&quot;:{&quot;artist&q
>>>>> uot;:&quot;Prince&quot;,&quot;songtitle&quot;:&quot;Never
Let Me
>>>>> Down&quot;}}]</
>>>>> metadata>
>>>>> <author><nickname>imtiaz2</nickname><id>3</id></author>
>>>>> <tags></tags><replyto></replyto><conversation></conversation>
>>>>> </message></api>
>>>>>
>>>>> imtiaz@imtiaz-20100131 /cygdrive/d/temp
>>>>> $
>>>>>
>>>>> Imtiaz
>>>>>
>>>>> ----- Original Message ----- From: "Ethan Jewett" <esjewett@gmail.com>
>>>>> To: <esme-dev@incubator.apache.org>; <in.imtiaz@gmail.com>
>>>>> Sent: Saturday, July 17, 2010 8:30 PM
>>>>> Subject: Re: Release planning
>>>>>
>>>>>
>>>>> Re ESME-242 - I think you (and Vassil) are right, but I haven't been
>>>>> able to independently confirm and I haven't seen a working API call
>>>>> attached to the Jira issue, so I'm not willing to close it yet. If we
>>>>> had working metadata examples using properly escaped XML and JSON, I
>>>>> would be very happy :-)
>>>>>
>>>>> Ethan
>>>>>
>>>>> On Sat, Jul 17, 2010 at 4:32 PM, <in.imtiaz@gmail.com> wrote:
>>>>>>
>>>>>> About ESME-242,
>>>>>> Per Vassil Dichev's mail to this group that http POST text used in
the
>>>>>> shell curl command must not be quoted and must be url encoded or
must
>>>>>> be
>>>>>> quoted using single quotes, this Jira issue should be resolved to
'not
>>>>>> a
>>>>>> bug', if I recall correctly, that is!
>>>>>> Imtiaz
>>>>>>
>>>>>> Sent from BlackBerryĹ˝ on Airtel
>>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Ethan Jewett <esjewett@gmail.com>
>>>>>> Date: Sat, 17 Jul 2010 16:11:44
>>>>>> To: <esme-dev@incubator.apache.org>
>>>>>> Reply-To: esme-dev@incubator.apache.org
>>>>>> Subject: Re: Release planning
>>>>>>
>>>>>> Thoughts on items remaining for this release:
>>>>>>
>>>>>> ESME-225 - "need unit test for GET api2/users/actions don't retrieve
>>>>>> disabled actions" - I don't think this is required for release and
>>>>>> should be moved to the backlog.
>>>>>>
>>>>>> ESME-242 - "The use of metadata in the api2 currently doesn't work"
-
>>>>>> I think this is important but also not required for release and should
>>>>>> be moved to the backlog.
>>>>>>
>>>>>> I agree that ESME-205 and ESME-212 are major. ESME-212 at least is
a
>>>>>> blocker.
>>>>>>
>>>>>>
>>>>>>
>>>>>> Regarding changes to be made to API2: I've created JIRA issues for
all
>>>>>> the todos at the bottom of the file (except a couple that were already
>>>>>> done). It was lazy of me to put a todo-list in the file - I should
>>>>>> really avoid that and force myself to have everything in Jira. I
will
>>>>>> be checking in a version of the file without the todo-list shortly.
>>>>>>
>>>>>>
>>>>>> Cheers,
>>>>>> Ethan
>>>>>>
>>>>>> On Fri, Jul 16, 2010 at 12:40 PM, Richard Hirsch
>>>>>> <hirsch.dick@gmail.com>
>>>>>> wrote:
>>>>>>>
>>>>>>> I've been working on the JIRA items and we have 10 items left
for
>>>>>>> this
>>>>>>> release.
>>>>>>>
>>>>>>> The following items are linked to IE 7:
>>>>>>> * ESME-207
>>>>>>> * ESME-239
>>>>>>> * ESME-208
>>>>>>>
>>>>>>> Some of these items are CSS-related (for example, 207) but the
other
>>>>>>> ones are javascript / jquery related.
>>>>>>>
>>>>>>> I think the two main issues are:
>>>>>>>
>>>>>>> * ESME-205 Search is broken (This only happens when you use a
JDBC
>>>>>>> based compass indexes - first noticed on Stax but it also is
present
>>>>>>> locally)
>>>>>>> * ESME-212 Messages from pools aren't hidden
>>>>>>>
>>>>>>> I'm going to be gone the next four weeks on vacation, so either
>>>>>>> someone else can coordinate the release or we wait until I return
to
>>>>>>> continue with our release preparations.
>>>>>>>
>>>>>>> If anyone wants to work on other items until the release, here
are
>>>>>>> other items that I consider important:
>>>>>>>
>>>>>>> * ESME-108 - View my pools" functionality
>>>>>>> * ESME-170 Pubsubhubbub support for Atom & RSS actions#
>>>>>>> * ESME-214 Add container-based authentication
>>>>>>> * ESME-213 Twitter API is out-of-date and API calls don't always
>>>>>>> return the correct info
>>>>>>>
>>>>>>> -- other api2 related changes - see the ToDos at the bottom of
the
>>>>>>> API2.scala page .
>>>>>>>
>>>>>>> D.
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>
>
>

Mime
View raw message