Return-Path: Delivered-To: apmail-incubator-esme-dev-archive@minotaur.apache.org Received: (qmail 32547 invoked from network); 23 Jul 2010 09:23:27 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 23 Jul 2010 09:23:27 -0000 Received: (qmail 623 invoked by uid 500); 23 Jul 2010 09:23:27 -0000 Delivered-To: apmail-incubator-esme-dev-archive@incubator.apache.org Received: (qmail 555 invoked by uid 500); 23 Jul 2010 09:23:24 -0000 Mailing-List: contact esme-dev-help@incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: esme-dev@incubator.apache.org Delivered-To: mailing list esme-dev@incubator.apache.org Received: (qmail 547 invoked by uid 99); 23 Jul 2010 09:23:24 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 23 Jul 2010 09:23:24 +0000 X-ASF-Spam-Status: No, hits=0.0 required=10.0 tests=FREEMAIL_FROM,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of yojibee@gmail.com designates 209.85.161.47 as permitted sender) Received: from [209.85.161.47] (HELO mail-fx0-f47.google.com) (209.85.161.47) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 23 Jul 2010 09:23:14 +0000 Received: by fxm12 with SMTP id 12so4964405fxm.6 for ; Fri, 23 Jul 2010 02:22:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:subject:mime-version :content-type:from:x-priority:in-reply-to:date :content-transfer-encoding:message-id:references:to:x-mailer; bh=Ykm3LxenPUKrDQt3Bove+FhDpkNjSu2yY/RyA/6nZYg=; b=h3pmgOsPsHmPHpLGsLpH/p3WKI/2i89SzYhMwP8+un2Zcy8JMNIdfgd5fWw/bqkOlp mrR/eheo+bFaafPfFAmJHGkc9FBmgSxDWGjoiyDRe/xNTeKrXIq0ZPSWF5oTSbna1u78 sLvzywaVQrJm4ykv+8vTN1RxVerMa0q9vscSo= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=subject:mime-version:content-type:from:x-priority:in-reply-to:date :content-transfer-encoding:message-id:references:to:x-mailer; b=bH7O6fwUN5g16qS3MLMcI561AFY8CwMPLDYlr8v96+CQw9M2cwD+RPZhZeM+r4OKvA gAqpVOhd6w2cm4d8abfropY686FvDV0cOF0rhCSygLnKuBV2clW/mN9xb8i9CaSblNyw XjKL97goWvLoPB0M+vYF7UO/78QdMhr2PgtDc= Received: by 10.223.113.144 with SMTP id a16mr3053245faq.41.1279876973288; Fri, 23 Jul 2010 02:22:53 -0700 (PDT) Received: from [192.168.175.33] (port-83-236-197-180.static.qsc.de [83.236.197.180]) by mx.google.com with ESMTPS id b11sm13610faq.6.2010.07.23.02.22.51 (version=TLSv1/SSLv3 cipher=RC4-MD5); Fri, 23 Jul 2010 02:22:52 -0700 (PDT) Subject: Re: Metadata handling (was "Release planning") Mime-Version: 1.0 (Apple Message framework v1078) Content-Type: text/plain; charset=us-ascii From: =?iso-8859-1?Q?Anne_Kathrine_Petter=F8e?= X-Priority: 3 In-Reply-To: <03A5F4A374134EBFAE643DA57A9B6465@imtiaz20100131> Date: Fri, 23 Jul 2010 11:22:58 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <0E328AC4-5CCF-472F-A4F8-F31738E1839D@gmail.com> References: <03A5F4A374134EBFAE643DA57A9B6465@imtiaz20100131> To: esme-dev@incubator.apache.org X-Mailer: Apple Mail (2.1078) X-Virus-Checked: Checked by ClamAV on apache.org Oh and sorry for being a bit out of touch again, getting into things at = Adobe was more time consuming than I had expected. Should be back = on-list over the weekend and next week. On 23. juli 2010, at 11.03, Imtiaz Ahmed H E wrote: > Maybe I'm just cc'ing myself too !!! > Sorry! >=20 > ----- Original Message ----- From: "Imtiaz Ahmed H E" = > To: > Sent: Friday, July 23, 2010 2:30 PM > Subject: Re: Metadata handling (was "Release planning") >=20 >=20 >> By the way, someone please tell me why I have started receiving my = own posts to the lists also since the last couple or so of posts. It's = pretty irritating :) >>=20 >> Imtiaz >>=20 >> ----- Original Message ----- From: "Imtiaz Ahmed H E" = >> To: >> Sent: Friday, July 23, 2010 2:24 PM >> Subject: Re: Metadata handling (was "Release planning") >>=20 >>=20 >>> I have attached a fix (Esme_branches_metadata_Jira242Fix.diff) to = Jira ticket 242, with the following change >>>=20 >>> The line, >>>=20 >>> lazy val metadata: String =3D originalXml \ "metadata" text >>>=20 >>> commented and replaced by the line >>>=20 >>> lazy val metadata: String =3D (originalXml \ "metadata").toString >>>=20 >>> See my earlier mail to esme-dev for results from the fix. >>>=20 >>> The fix is only for the metadata branch in the SVN repo directory = 'branches'. >>>=20 >>> This is pending further evaluation by esme-dev and improvements = etc... >>>=20 >>> Imtiaz >>>=20 >>> ----- Original Message ----- From: "Ethan Jewett" = >>> To: >>> Sent: Wednesday, July 21, 2010 7:51 PM >>> Subject: Re: Metadata handling (was "Release planning") >>>=20 >>>=20 >>>> Sounds good to me, at least until we can figure out what the next >>>> step/problem is on the metadata front. Just ping the list when = you've >>>> uploaded the patch. >>>>=20 >>>> You don't happen to have any JDBC/JNDI and/or Lucene experience do >>>> you? I'm totally stumped on the issue with search when we have it = run >>>> using the database instead of the filesystem (ESME-205). >>>>=20 >>>> Ethan >>>>=20 >>>> On Wed, Jul 21, 2010 at 4:06 PM, Imtiaz Ahmed H E = wrote: >>>>> Oops! I think I wrote that in a hurry, not even thinking!! >>>>>=20 >>>>> Actually, I think I should just submit a patch to the metadata = branch with >>>>> my current change and move on to other Jira tickets while we wait = for >>>>> consensus/finalization. >>>>>=20 >>>>> Imtiaz >>>>>=20 >>>>> ----- Original Message ----- From: "Imtiaz Ahmed H E" = >>>>> To: >>>>> Sent: Wednesday, July 21, 2010 7:30 PM >>>>> Subject: Re: Metadata handling (was "Release planning") >>>>>=20 >>>>>=20 >>>>>> How about it if I just store the metadata as a literal string and = return >>>>>> it as such for now and submit a patch to the metadata branch with = that and >>>>>> move on to another Jira ticket ? >>>>>>=20 >>>>>> Then, when we all agree on something final for the output format = and >>>>>> representation of message metadata we can look at further work on = ESME-242. >>>>>>=20 >>>>>> Imtiaz >>>>>>=20 >>>>>> ----- Original Message ----- From: "Ethan Jewett" = >>>>>> To: >>>>>> Sent: Wednesday, July 21, 2010 7:01 PM >>>>>> Subject: Re: Metadata handling (was "Release planning") >>>>>>=20 >>>>>>=20 >>>>>> Imtiaz, >>>>>>=20 >>>>>> Excellent! That's good progress. >>>>>>=20 >>>>>> Any idea what the API response is still getting HTML = entity-encoded? >>>>>> That's what all of those < and > things are. Does the API = do >>>>>> this for all of its responses or is it just the metadata ones? We >>>>>> probably don't need to be entity-encoding since we're returning = XML, >>>>>> not HTML. >>>>>>=20 >>>>>> Also, I just created a branch called "metadata". Can you work = against >>>>>> this branch and feel free to submit patches against that branch = to the >>>>>> Jira ticket. Vassil, Anne, or I can then apply the patches = directly to >>>>>> the branch without the level of review that would be needed for a >>>>>> patch against trunk. This should help allow us to try out your = changes >>>>>> and come to consensus and understanding more quickly. >>>>>>=20 >>>>>> Thanks! >>>>>> Ethan >>>>>>=20 >>>>>> On Wed, Jul 21, 2010 at 7:20 AM, Imtiaz Ahmed H E = >>>>>> wrote: >>>>>>>=20 >>>>>>> By the way, all I did was, >>>>>>>=20 >>>>>>> in Message.scala, >>>>>>> replace, >>>>>>>=20 >>>>>>> lazy val metadata: String =3D originalXml \ "metadata" text >>>>>>>=20 >>>>>>> by... >>>>>>>=20 >>>>>>> lazy val metadata: String =3D (originalXml \ = "metadata").toString >>>>>>>=20 >>>>>>> to get the results you see in my previous mail... >>>>>>>=20 >>>>>>> Imtiaz >>>>>>>=20 >>>>>>> ----- Original Message ----- From: "Imtiaz Ahmed H E" >>>>>>> >>>>>>> To: >>>>>>> Sent: Wednesday, July 21, 2010 10:34 AM >>>>>>> Subject: Re: Metadata handling (was "Release planning") >>>>>>>=20 >>>>>>>=20 >>>>>>>> Hi, >>>>>>>>=20 >>>>>>>> In the absence of decisive consensus yet, I made a small change = in >>>>>>>> Message.scala as a first step in fixing ESME-242. >>>>>>>> Here's curl output for the Jira ticket examples given there by >>>>>>>> Ethan...There is maybe obvious unacceptable stuff here, but let = me >>>>>>>> know... >>>>>>>>=20 >>>>>>>> imtiaz@imtiaz-20100131 /cygdrive/d/temp >>>>>>>> $ curl --dump-header headers -d = "token=3DHEZTQKM525SAMIPN4EDVRUOGHI40AKBL" >>>>>>>> http:/ >>>>>>>> = /localhost:8080/esme-server-apache-esme-1.0-RC1-incubating/api2/session >>>>>>>> >>>>>>>>=20 >>>>>>>>=20 >>>>>>>> = 3imtiaz2None>>>>>>> ole_name>I A 2 H E >>>>>>>>=20 >>>>>>>> imtiaz@imtiaz-20100131 /cygdrive/d/temp >>>>>>>> $ cat headers >>>>>>>> HTTP/1.1 200 OK >>>>>>>> Server: Apache-Coyote/1.1 >>>>>>>> Set-Cookie: JSESSIONID=3DC7B688BAF8E99B2A638EF432885E310E; >>>>>>>> Path=3D/esme-server-apach >>>>>>>> e-esme-1.0-RC1-incubating >>>>>>>> Expires: Wed, 21 Jul 2010 04:56:51 UTC >>>>>>>> Date: Wed, 21 Jul 2010 04:56:51 GMT >>>>>>>> Pragma: no-cache >>>>>>>> Cache-Control: no-cache; private; no-store >>>>>>>> X-Lift-Version: 2.0-SNAPSHOT >>>>>>>> Content-Type: text/xml;charset=3Dutf-8 >>>>>>>> Content-Length: 178 >>>>>>>>=20 >>>>>>>>=20 >>>>>>>> imtiaz@imtiaz-20100131 /cygdrive/d/temp >>>>>>>> $ curl -b headers -d >>>>>>>> 'message=3Dtest200&metadata=3DHello>>>>>>> tameta>Meta' >>>>>>>> http://localhost:8080/esme-ser >>>>>>>> ver-apache-esme-1.0-RC1-incubating/api2/user/messages >>>>>>>> >>>>>>>> >>>>>>>> 28 >>>>>>>> Wed, 21 Jul 2010 04:57:05 UTC >>>>>>>> api2 >>>>>>>> test200 >>>>>>>>=20 >>>>>>>>=20 >>>>>>>>=20 >>>>>>>> = <metadata><outer><meta><metameta>Hello&l= t;/m >>>>>>>>=20 >>>>>>>>=20 >>>>>>>> = etameta></meta><onlymeta>Meta</onlymeta></outer>= ;</ >>>>>>>> metadata> >>>>>>>> imtiaz23 >>>>>>>> >>>>>>>> >>>>>>>>=20 >>>>>>>> imtiaz@imtiaz-20100131 /cygdrive/d/temp >>>>>>>> $ curl -b headers -d >>>>>>>> 'message=3Dtest201&metadata=3D"meta":[{"place":{"place >>>>>>>>=20 >>>>>>>>=20 >>>>>>>> = _type":"city","region":"CA+"}},{"song":{"artist":"Prince","songtitle":"Nev= er+Le >>>>>>>> t+Me+Down"}}]' >>>>>>>> http://localhost:8080/esme-server-apache-esme-1.0-RC1-i >>>>>>>> ncubating/api2/user/messages >>>>>>>> >>>>>>>> >>>>>>>> 29 >>>>>>>> Wed, 21 Jul 2010 04:57:39 UTC >>>>>>>> api2 >>>>>>>> test201 >>>>>>>>=20 >>>>>>>>=20 >>>>>>>>=20 >>>>>>>> = <metadata><anytag>&quot;meta&quot;:[{&qu= ot;p >>>>>>>>=20 >>>>>>>>=20 >>>>>>>> = lace&quot;:{&quot;place_type&quot;:&quot;city&quot;,&a= mp;quo >>>>>>>> t;region&quot;:&quot;CA >>>>>>>> &quot;}},{&quot;song&quot;:{&quo >>>>>>>>=20 >>>>>>>>=20 >>>>>>>> = t;artist&quot;:&quot;Prince&quot;,&quot;songtitle&quot= ;:& >>>>>>>> ;quot;Never Let Me >>>>>>>> Down&quot;}}]</anytag></metadata> >>>>>>>>=20 >>>>>>>> imtiaz23 >>>>>>>> >>>>>>>> >>>>>>>>=20 >>>>>>>> imtiaz@imtiaz-20100131 /cygdrive/d/temp >>>>>>>> $ curl -b headers -d >>>>>>>> 'message=3Dtest202&metadata=3D"meta":[{"place":{"place_type":" >>>>>>>>=20 >>>>>>>>=20 >>>>>>>> = city","region":"CA+"}},{"song":{"artist":"Prince","songtitle":"Never+Let+M= e+Dow >>>>>>>> n"}}]' >>>>>>>> = http://localhost:8080/esme-server-apache-esme-1.0-RC1-incubating/api2/us >>>>>>>> er/messages >>>>>>>> >>>>>>>> >>>>>>>> 30 >>>>>>>> Wed, 21 Jul 2010 04:58:02 UTC >>>>>>>> api2 >>>>>>>> test202 >>>>>>>> >>>>>>>> imtiaz23 >>>>>>>> >>>>>>>> >>>>>>>>=20 >>>>>>>> imtiaz@imtiaz-20100131 /cygdrive/d/temp >>>>>>>> $ curl -b headers >>>>>>>> http://localhost:8080/esme-server-apache-esme-1.0-RC1-incubat >>>>>>>> ing/api2/user/messages >>>>>>>> >>>>>>>> >>>>>>>> 30 >>>>>>>> Wed, 21 Jul 2010 04:58:02 UTC >>>>>>>> api2 >>>>>>>> test202 >>>>>>>> >>>>>>>> imtiaz23 >>>>>>>> >>>>>>>> >>>>>>>> 29 >>>>>>>> Wed, 21 Jul 2010 04:57:39 UTC >>>>>>>> api2 >>>>>>>> test201 >>>>>>>>=20 >>>>>>>>=20 >>>>>>>>=20 >>>>>>>> = <metadata><anytag>&quot;meta&quot;:[{&qu= ot;p >>>>>>>>=20 >>>>>>>>=20 >>>>>>>> = lace&quot;:{&quot;place_type&quot;:&quot;city&quot;,&a= mp;quo >>>>>>>> t;region&quot;:&quot;CA >>>>>>>> &quot;}},{&quot;song&quot;:{&quo >>>>>>>>=20 >>>>>>>>=20 >>>>>>>> = t;artist&quot;:&quot;Prince&quot;,&quot;songtitle&quot= ;:& >>>>>>>> ;quot;Never Let Me >>>>>>>> Down&quot;}}]</anytag></metadata> >>>>>>>>=20 >>>>>>>> imtiaz23 >>>>>>>> >>>>>>>> >>>>>>>> 28 >>>>>>>> Wed, 21 Jul 2010 04:57:05 UTC >>>>>>>> api2 >>>>>>>> test200 >>>>>>>>=20 >>>>>>>>=20 >>>>>>>>=20 >>>>>>>> = <metadata><outer><meta><metameta>Hello&l= t;/m >>>>>>>>=20 >>>>>>>>=20 >>>>>>>> = etameta></meta><onlymeta>Meta</onlymeta></outer>= ;</ >>>>>>>> metadata> >>>>>>>> imtiaz23 >>>>>>>> >>>>>>>> >>>>>>>>=20 >>>>>>>> imtiaz@imtiaz-20100131 /cygdrive/d/temp >>>>>>>> $ >>>>>>>>=20 >>>>>>>> ----- Original Message ----- From: "Vassil Dichev" = >>>>>>>> To: >>>>>>>> Sent: Tuesday, July 20, 2010 11:27 AM >>>>>>>> Subject: Re: Metadata handling (was "Release planning") >>>>>>>>=20 >>>>>>>>=20 >>>>>>>>>> I'd prefer option 1 (separate attribute from text). Within = this >>>>>>>>>> separate attribute there is the question of how data is >>>>>>>>>> stored/represented. I'm ok with either raw string or a = tuple-based >>>>>>>>>> structure like Twitter's. I kind of like the tuple = (key-value) >>>>>>>>>> approach. >>>>>>>>>=20 >>>>>>>>> I'm not too interested in how the data is stored, because it's = fairly >>>>>>>>> trivial to implement either way. It's currently not yet clear = to me >>>>>>>>> what the requirements for the output format are. >>>>>>>>>=20 >>>>>>>>>> What I'm insisting on and what I was saying we got wrong is = that what >>>>>>>>>> goes in needs to be the same as what comes out. If it's = tuple-based >>>>>>>>>> and I send in a tuple, then I should get that tuple (key and = value) >>>>>>>>>> back out when I request the metadata for a message. Right now = I think >>>>>>>>>> we only get a concatenated list of values from the metadata = and >>>>>>>>>> metaData methods and we're bound to an XML format. >>>>>>>>>=20 >>>>>>>>> I don't get it. A tuple is an abstraction which might be = expressed in >>>>>>>>> a specific format. So what goes in is not what comes out = depending on >>>>>>>>> the format. Let me quote the specific example Twitter = provides. >>>>>>>>>=20 >>>>>>>>> This comes in: >>>>>>>>>=20 >>>>>>>>> "annotations": >>>>>>>>> [{"type":{"another_attribute":"value", "attribute":"value"}}] >>>>>>>>>=20 >>>>>>>>> This comes out: >>>>>>>>>=20 >>>>>>>>> >>>>>>>>> >>>>>>>>> foo >>>>>>>>> >>>>>>>>> >>>>>>>>> bar >>>>>>>>> baz >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>=20 >>>>>>>>>=20 >>>>>>>>>> As far as requiring a particular format, I think the internal = format >>>>>>>>>> should be either a raw string or a immutable hashmap with raw = strings >>>>>>>>>> as keys and values. We can handle converting this to XML or = JSON in >>>>>>>>>> the API or view code. >>>>>>>>>=20 >>>>>>>>> Again, it's not so interesting what's internally there, let's = just >>>>>>>>> treat it as a black box. What I want to know is, do we want to = have >>>>>>>>> for instance XML in a JSON reply returned: >>>>>>>>>=20 >>>>>>>>> "annotations": >>>>>>>>> [{"type":{" bar >>>>>>>>> baz "}}] >>>>>>>>>=20 >>>>>>>>> or, inversely, do we want JSON inside an XML reply? Something = like: >>>>>>>>>=20 >>>>>>>>> >>>>>>>>> >>>>>>>>> foo >>>>>>>>> >>>>>>>>> {"another_attribute":"value", "attribute":"value"} >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>=20 >>>>>>>>> because the latter will need to be escaped. >>>>>>>>>=20 >>>>>>>>> Sorry for being too dense, but in any case, we either have to = escape >>>>>>>>> the metadata or we have to transform the structure to XML/JSON = when we >>>>>>>>> return it back to the user. None of these is "what goes in = needs to be >>>>>>>>> the same as what comes out" >>>>>>>>=20 >>>>>>>=20 >>>>>>>=20 >>>>>>=20 >>>>>=20 >>>>>=20 >>>=20 >=20