incubator-esme-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ethan Jewett <esjew...@gmail.com>
Subject Re: Metadata handling (was "Release planning")
Date Wed, 21 Jul 2010 14:21:16 GMT
Sounds good to me, at least until we can figure out what the next
step/problem is on the metadata front. Just ping the list when you've
uploaded the patch.

You don't happen to have any JDBC/JNDI and/or Lucene experience do
you? I'm totally stumped on the issue with search when we have it run
using the database instead of the filesystem (ESME-205).

Ethan

On Wed, Jul 21, 2010 at 4:06 PM, Imtiaz Ahmed H E <in.imtiaz@gmail.com> wrote:
> Oops! I think I wrote that in a hurry, not even thinking!!
>
> Actually, I think I should just submit a patch to the metadata branch with
> my current change and move on to other Jira tickets while we wait for
> consensus/finalization.
>
> Imtiaz
>
> ----- Original Message ----- From: "Imtiaz Ahmed H E" <in.imtiaz@gmail.com>
> To: <esme-dev@incubator.apache.org>
> Sent: Wednesday, July 21, 2010 7:30 PM
> Subject: Re: Metadata handling (was "Release planning")
>
>
>> How about it if I just store the metadata as a literal string and return
>> it as such for now and submit a patch to the metadata branch with that and
>> move on to another Jira ticket ?
>>
>> Then, when we all agree on something final for the output format and
>> representation of message metadata we can look at further work on ESME-242.
>>
>> Imtiaz
>>
>> ----- Original Message ----- From: "Ethan Jewett" <esjewett@gmail.com>
>> To: <esme-dev@incubator.apache.org>
>> Sent: Wednesday, July 21, 2010 7:01 PM
>> Subject: Re: Metadata handling (was "Release planning")
>>
>>
>> Imtiaz,
>>
>> Excellent! That's good progress.
>>
>> Any idea what the API response is still getting HTML entity-encoded?
>> That's what all of those &lt; and &gt; things are. Does the API do
>> this for all of its responses or is it just the metadata ones? We
>> probably don't need to be entity-encoding since we're returning XML,
>> not HTML.
>>
>> Also, I just created a branch called "metadata". Can you work against
>> this branch and feel free to submit patches against that branch to the
>> Jira ticket. Vassil, Anne, or I can then apply the patches directly to
>> the branch without the level of review that would be needed for a
>> patch against trunk. This should help allow us to try out your changes
>> and come to consensus and understanding more quickly.
>>
>> Thanks!
>> Ethan
>>
>> On Wed, Jul 21, 2010 at 7:20 AM, Imtiaz Ahmed H E <in.imtiaz@gmail.com>
>> wrote:
>>>
>>> By the way, all I did was,
>>>
>>> in Message.scala,
>>> replace,
>>>
>>> lazy val metadata: String = originalXml \ "metadata" text
>>>
>>> by...
>>>
>>> lazy val metadata: String = (originalXml \ "metadata").toString
>>>
>>> to get the results you see in my previous mail...
>>>
>>> Imtiaz
>>>
>>> ----- Original Message ----- From: "Imtiaz Ahmed H E"
>>> <in.imtiaz@gmail.com>
>>> To: <esme-dev@incubator.apache.org>
>>> Sent: Wednesday, July 21, 2010 10:34 AM
>>> Subject: Re: Metadata handling (was "Release planning")
>>>
>>>
>>>> Hi,
>>>>
>>>> In the absence of decisive consensus yet, I made a small change in
>>>> Message.scala as a first step in fixing ESME-242.
>>>> Here's curl output for the Jira ticket examples given there by
>>>> Ethan...There is maybe obvious unacceptable stuff here, but let me
>>>> know...
>>>>
>>>> imtiaz@imtiaz-20100131 /cygdrive/d/temp
>>>> $ curl --dump-header headers -d "token=HEZTQKM525SAMIPN4EDVRUOGHI40AKBL"
>>>> http:/
>>>> /localhost:8080/esme-server-apache-esme-1.0-RC1-incubating/api2/session
>>>> <?xml version="1.0" encoding="UTF-8"?>
>>>>
>>>>
>>>> <api><session><user><id>3</id><nickname>imtiaz2</nickname><image>None</image><wh
>>>> ole_name>I A 2 H E</whole_name></user></session></api>
>>>>
>>>> imtiaz@imtiaz-20100131 /cygdrive/d/temp
>>>> $ cat headers
>>>> HTTP/1.1 200 OK
>>>> Server: Apache-Coyote/1.1
>>>> Set-Cookie: JSESSIONID=C7B688BAF8E99B2A638EF432885E310E;
>>>> Path=/esme-server-apach
>>>> e-esme-1.0-RC1-incubating
>>>> Expires: Wed, 21 Jul 2010 04:56:51 UTC
>>>> Date: Wed, 21 Jul 2010 04:56:51 GMT
>>>> Pragma: no-cache
>>>> Cache-Control: no-cache; private; no-store
>>>> X-Lift-Version: 2.0-SNAPSHOT
>>>> Content-Type: text/xml;charset=utf-8
>>>> Content-Length: 178
>>>>
>>>>
>>>> imtiaz@imtiaz-20100131 /cygdrive/d/temp
>>>> $ curl -b headers -d
>>>> 'message=test200&metadata=<outer><meta><metameta>Hello</me
>>>> tameta></meta><onlymeta>Meta</onlymeta></outer>'
>>>> http://localhost:8080/esme-ser
>>>> ver-apache-esme-1.0-RC1-incubating/api2/user/messages
>>>> <?xml version="1.0" encoding="UTF-8"?>
>>>> <api><message>
>>>> <id>28</id>
>>>> <date>Wed, 21 Jul 2010 04:57:05 UTC</date>
>>>> <source>api2</source>
>>>> <body>test200</body>
>>>>
>>>>
>>>>
>>>> <metadata>&lt;metadata&gt;&lt;outer&gt;&lt;meta&gt;&lt;metameta&gt;Hello&lt;/m
>>>>
>>>>
>>>> etameta&gt;&lt;/meta&gt;&lt;onlymeta&gt;Meta&lt;/onlymeta&gt;&lt;/outer&gt;&lt;/
>>>> metadata&gt;</metadata>
>>>> <author><nickname>imtiaz2</nickname><id>3</id></author>
>>>> <tags></tags><replyto></replyto><conversation></conversation>
>>>> </message></api>
>>>>
>>>> imtiaz@imtiaz-20100131 /cygdrive/d/temp
>>>> $ curl -b headers -d
>>>> 'message=test201&metadata=<anytag>"meta":[{"place":{"place
>>>>
>>>>
>>>> _type":"city","region":"CA+"}},{"song":{"artist":"Prince","songtitle":"Never+Le
>>>> t+Me+Down"}}]</anytag>'
>>>> http://localhost:8080/esme-server-apache-esme-1.0-RC1-i
>>>> ncubating/api2/user/messages
>>>> <?xml version="1.0" encoding="UTF-8"?>
>>>> <api><message>
>>>> <id>29</id>
>>>> <date>Wed, 21 Jul 2010 04:57:39 UTC</date>
>>>> <source>api2</source>
>>>> <body>test201</body>
>>>>
>>>>
>>>>
>>>> <metadata>&lt;metadata&gt;&lt;anytag&gt;&amp;quot;meta&amp;quot;:[{&amp;quot;p
>>>>
>>>>
>>>> lace&amp;quot;:{&amp;quot;place_type&amp;quot;:&amp;quot;city&amp;quot;,&amp;quo
>>>> t;region&amp;quot;:&amp;quot;CA
>>>> &amp;quot;}},{&amp;quot;song&amp;quot;:{&amp;quo
>>>>
>>>>
>>>> t;artist&amp;quot;:&amp;quot;Prince&amp;quot;,&amp;quot;songtitle&amp;quot;:&amp
>>>> ;quot;Never Let Me
>>>> Down&amp;quot;}}]&lt;/anytag&gt;&lt;/metadata&gt;</metadata>
>>>>
>>>> <author><nickname>imtiaz2</nickname><id>3</id></author>
>>>> <tags></tags><replyto></replyto><conversation></conversation>
>>>> </message></api>
>>>>
>>>> imtiaz@imtiaz-20100131 /cygdrive/d/temp
>>>> $ curl -b headers -d
>>>> 'message=test202&metadata="meta":[{"place":{"place_type":"
>>>>
>>>>
>>>> city","region":"CA+"}},{"song":{"artist":"Prince","songtitle":"Never+Let+Me+Dow
>>>> n"}}]'
>>>> http://localhost:8080/esme-server-apache-esme-1.0-RC1-incubating/api2/us
>>>> er/messages
>>>> <?xml version="1.0" encoding="UTF-8"?>
>>>> <api><message>
>>>> <id>30</id>
>>>> <date>Wed, 21 Jul 2010 04:58:02 UTC</date>
>>>> <source>api2</source>
>>>> <body>test202</body>
>>>> <metadata></metadata>
>>>> <author><nickname>imtiaz2</nickname><id>3</id></author>
>>>> <tags></tags><replyto></replyto><conversation></conversation>
>>>> </message></api>
>>>>
>>>> imtiaz@imtiaz-20100131 /cygdrive/d/temp
>>>> $ curl -b headers
>>>> http://localhost:8080/esme-server-apache-esme-1.0-RC1-incubat
>>>> ing/api2/user/messages
>>>> <?xml version="1.0" encoding="UTF-8"?>
>>>> <api><messages><message>
>>>> <id>30</id>
>>>> <date>Wed, 21 Jul 2010 04:58:02 UTC</date>
>>>> <source>api2</source>
>>>> <body>test202</body>
>>>> <metadata></metadata>
>>>> <author><nickname>imtiaz2</nickname><id>3</id></author>
>>>> <tags></tags><replyto></replyto><conversation></conversation>
>>>> </message><message>
>>>> <id>29</id>
>>>> <date>Wed, 21 Jul 2010 04:57:39 UTC</date>
>>>> <source>api2</source>
>>>> <body>test201</body>
>>>>
>>>>
>>>>
>>>> <metadata>&lt;metadata&gt;&lt;anytag&gt;&amp;quot;meta&amp;quot;:[{&amp;quot;p
>>>>
>>>>
>>>> lace&amp;quot;:{&amp;quot;place_type&amp;quot;:&amp;quot;city&amp;quot;,&amp;quo
>>>> t;region&amp;quot;:&amp;quot;CA
>>>> &amp;quot;}},{&amp;quot;song&amp;quot;:{&amp;quo
>>>>
>>>>
>>>> t;artist&amp;quot;:&amp;quot;Prince&amp;quot;,&amp;quot;songtitle&amp;quot;:&amp
>>>> ;quot;Never Let Me
>>>> Down&amp;quot;}}]&lt;/anytag&gt;&lt;/metadata&gt;</metadata>
>>>>
>>>> <author><nickname>imtiaz2</nickname><id>3</id></author>
>>>> <tags></tags><replyto></replyto><conversation></conversation>
>>>> </message><message>
>>>> <id>28</id>
>>>> <date>Wed, 21 Jul 2010 04:57:05 UTC</date>
>>>> <source>api2</source>
>>>> <body>test200</body>
>>>>
>>>>
>>>>
>>>> <metadata>&lt;metadata&gt;&lt;outer&gt;&lt;meta&gt;&lt;metameta&gt;Hello&lt;/m
>>>>
>>>>
>>>> etameta&gt;&lt;/meta&gt;&lt;onlymeta&gt;Meta&lt;/onlymeta&gt;&lt;/outer&gt;&lt;/
>>>> metadata&gt;</metadata>
>>>> <author><nickname>imtiaz2</nickname><id>3</id></author>
>>>> <tags></tags><replyto></replyto><conversation></conversation>
>>>> </message></messages></api>
>>>>
>>>> imtiaz@imtiaz-20100131 /cygdrive/d/temp
>>>> $
>>>>
>>>> ----- Original Message ----- From: "Vassil Dichev" <vdichev@apache.org>
>>>> To: <esme-dev@incubator.apache.org>
>>>> Sent: Tuesday, July 20, 2010 11:27 AM
>>>> Subject: Re: Metadata handling (was "Release planning")
>>>>
>>>>
>>>>>> I'd prefer option 1 (separate attribute from text). Within this
>>>>>> separate attribute there is the question of how data is
>>>>>> stored/represented. I'm ok with either raw string or a tuple-based
>>>>>> structure like Twitter's. I kind of like the tuple (key-value)
>>>>>> approach.
>>>>>
>>>>> I'm not too interested in how the data is stored, because it's fairly
>>>>> trivial to implement either way. It's currently not yet clear to me
>>>>> what the requirements for the output format are.
>>>>>
>>>>>> What I'm insisting on and what I was saying we got wrong is that
what
>>>>>> goes in needs to be the same as what comes out. If it's tuple-based
>>>>>> and I send in a tuple, then I should get that tuple (key and value)
>>>>>> back out when I request the metadata for a message. Right now I think
>>>>>> we only get a concatenated list of values from the metadata and
>>>>>> metaData methods and we're bound to an XML format.
>>>>>
>>>>> I don't get it. A tuple is an abstraction which might be expressed in
>>>>> a specific format. So what goes in is not what comes out depending on
>>>>> the format. Let me quote the specific example Twitter provides.
>>>>>
>>>>> This comes in:
>>>>>
>>>>> "annotations":
>>>>> [{"type":{"another_attribute":"value", "attribute":"value"}}]
>>>>>
>>>>> This comes out:
>>>>>
>>>>> <annotations type="array">
>>>>> <annotation>
>>>>> <type>foo</type>
>>>>> <attributes>
>>>>> <attribute>
>>>>> <name>bar</name>
>>>>> <value>baz</value>
>>>>> </attribute>
>>>>> </attributes>
>>>>> </annotation>
>>>>> </annotations>
>>>>>
>>>>>
>>>>>> As far as requiring a particular format, I think the internal format
>>>>>> should be either a raw string or a immutable hashmap with raw strings
>>>>>> as keys and values. We can handle converting this to XML or JSON
in
>>>>>> the API or view code.
>>>>>
>>>>> Again, it's not so interesting what's internally there, let's just
>>>>> treat it as a black box. What I want to know is, do we want to have
>>>>> for instance XML in a JSON reply returned:
>>>>>
>>>>> "annotations":
>>>>> [{"type":{"<attributes> <attribute> <name>bar</name>
>>>>> <value>baz</value> </attribute> </attributes>"}}]
>>>>>
>>>>> or, inversely, do we want JSON inside an XML reply? Something like:
>>>>>
>>>>> <annotations type="array">
>>>>> <annotation>
>>>>> <type>foo</type>
>>>>> <attributes>
>>>>> {"another_attribute":"value", "attribute":"value"}
>>>>> </attributes>
>>>>> </annotation>
>>>>> </annotations>
>>>>>
>>>>> because the latter will need to be escaped.
>>>>>
>>>>> Sorry for being too dense, but in any case, we either have to escape
>>>>> the metadata or we have to transform the structure to XML/JSON when we
>>>>> return it back to the user. None of these is "what goes in needs to be
>>>>> the same as what comes out"
>>>>
>>>
>>>
>>
>
>

Mime
View raw message