Return-Path: Delivered-To: apmail-incubator-esme-dev-archive@minotaur.apache.org Received: (qmail 92964 invoked from network); 3 Sep 2010 05:28:21 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 3 Sep 2010 05:28:21 -0000 Received: (qmail 58784 invoked by uid 500); 3 Sep 2010 05:28:20 -0000 Delivered-To: apmail-incubator-esme-dev-archive@incubator.apache.org Received: (qmail 58725 invoked by uid 500); 3 Sep 2010 05:28:19 -0000 Mailing-List: contact esme-dev-help@incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: esme-dev@incubator.apache.org Delivered-To: mailing list esme-dev@incubator.apache.org Received: (qmail 58717 invoked by uid 99); 3 Sep 2010 05:28:18 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 03 Sep 2010 05:28:18 +0000 X-ASF-Spam-Status: No, hits=0.0 required=10.0 tests=FAKE_REPLY_C,FREEMAIL_FROM,RCVD_IN_DNSWL_NONE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of in.imtiaz@gmail.com designates 209.85.216.47 as permitted sender) Received: from [209.85.216.47] (HELO mail-qw0-f47.google.com) (209.85.216.47) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 03 Sep 2010 05:28:12 +0000 Received: by qwa26 with SMTP id 26so1330203qwa.6 for ; Thu, 02 Sep 2010 22:27:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:from:to:subject :date:mime-version:content-type:content-transfer-encoding:x-priority :x-msmail-priority:x-mailer:x-mimeole; bh=GgDRj1fL//ek2nNO7ZpdIXHQIg2mYArYlGBrgQSLc+I=; b=ZotnwaNFrF7ooVqVeyyKas0HU22UpdD3jkiwf56IsmGgQJh0+fGSeRM3aGffCAWaSH SUwFPsacVGGuxlFjS6e7H0G7Q/SYLlclWjzBB6Xwa1Q/ucCNBvfTPGKJxIlbeNXyeM3t nKD6OXD7p4cMm/GdWv89MKPRZVpwgCa4VGaK8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:from:to:subject:date:mime-version:content-type :content-transfer-encoding:x-priority:x-msmail-priority:x-mailer :x-mimeole; b=OYpRZIehH3T81qqj8aDmvB+mwsWtR6Az8Jn9FbuzstTk49xBbgM0H+DFB48xxrQpTX kl8t4e9fZp/StJLyuVk5t5XKdMKIdq35wgQYjbIMtxuWvPdbtyWOky0NgNW+qgIc2RaK oDhwaeV9czuwtr4jF/Kn9YvzMhSVdGjJ4ZJWM= Received: by 10.224.89.80 with SMTP id d16mr7473417qam.88.1283491671469; Thu, 02 Sep 2010 22:27:51 -0700 (PDT) Received: from imtiaz20100131 ([122.167.44.112]) by mx.google.com with ESMTPS id q8sm1468753qcs.0.2010.09.02.22.27.49 (version=SSLv3 cipher=RC4-MD5); Thu, 02 Sep 2010 22:27:51 -0700 (PDT) Message-ID: From: "Imtiaz Ahmed H E" To: Subject: Re: Fw: ESME - 266 - my fix in progress, hopefully... Date: Fri, 3 Sep 2010 10:57:45 +0530 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=response Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.5931 Thanks Vassil, the feed url picked up from the browser toolbar went through; But do let me know if we should be able to pick up feeds from the RSS button on the nytimes home page at the bottom, just as any Feed Reader app apparently/presumably does. Imtiaz ----- Original Message ----- From: "Imtiaz Ahmed H E" To: Sent: Friday, September 03, 2010 10:54 AM Subject: Re: Fw: ESME - 266 - my fix in progress, hopefully... > Okay, the Twitter feed went through when I started Esme a minute ago. > > Imtiaz > > ----- Original Message ----- > From: "Imtiaz Ahmed H E" > To: > Sent: Friday, September 03, 2010 10:50 AM > Subject: Re: Fw: ESME - 266 - my fix in progress, hopefully... > > >> Sorry, I posted the mail in the wrong thread. I have now changed the >> subject line. >> >> My concern is, "feed link got thro' right-click on RSS icon at bottom of >> nytimes.com and "copy link location" in windows context menu and paste" >> into say, FeedReader (perhaps any other feed reader app too) sets up the >> feed correctly. >> >> So, should we handle this in Esme too... CAN WE ? Should I? >> >> By the way what about the Twitter problem in the mail I sent along with >> the nytimes proble (see below). >> >> Imtiaz >> >> ----- Original Message ----- >> From: "Vassil Dichev" >> To: >> Sent: Friday, September 03, 2010 10:41 AM >> Subject: Re: ESME-267 - Pooled links in popular links list >> >> >> Well, it's obvious that if the XML couldn't parse the XML, then it's >> not valid XML. Add to this the fact that it contains a "meta" tag and >> it's clear that what's at the other end of the link was actually HTML. >> If you go to http://www.nytimes.com/rss, you'll notice this yourself. >> >> The real feed can be found by following the browser feed icon in the >> URL bar. It takes you to what the page describes as a feed in the tag >> . If you follow >> it, you will get to this URL: >> >> http://feeds.nytimes.com/nyt/rss/HomePage >> >> Try it, it should work. >> >> Vassil >> >> >> On Fri, Sep 3, 2010 at 2:35 AM, Imtiaz Ahmed H E >> wrote: >>> I have an action >>> >>> every 1 mins rss:http://www.nytimes.com/rss >>> >>> feed link got thro' right-click on RSS icon at bottom of nytimes.com and >>> "copy link location" in windows context menu and paste into esme with >>> rss: >>> prepended. >>> >>> and, >>> >>> every 1 mins >>> atom:http://twitter.com/statuses/user_timeline/esjewett.atom >>> >>> and I get in the Tomcat Window, >>> >>> WARN - Going to buffer response body of large or unknown size. Using >>> getResponse >>> BodyAsStream instead is recommended. >>> :37:31: expected closing tag of meta >>> >> href="http://www.nytimes.com/pages/sports/index.html">Sports< >>> /a> >>> ^ >>> >>> >>> and, apparently for the twitter feed, >>> >>> WARN - Going to buffer response body of large or unknown size. Using >>> getResponse >>> BodyAsStream instead is recommended. >>> :37:31: expected closing tag of meta >>> >> href="http://www.nytimes.com/pages/sports/index.html">Sports< >>> /a> >>> ^ >>> WARN - Cookie rejected: "$Version=0; k=122.167.31.233.1283470309076286; >>> $Path=/; >>> $Domain=.twitter.com". Illegal domain attribute ".twitter.com". Domain >>> of >>> origi >>> n: "twitter.com" >>> WARN - Cookie rejected: "$Version=0; >>> _twitter_sess=BAh7CDoPY3JlYXRlZF9hdGwrCNq2y >>> tQqAToHaWQiJWRjMTNkMjQ2ZGRiYjA2%250AOTU1ZGZjMTc1NjMxMTZhN2I4IgpmbGFzaElDOidBY3Rp >>> b25Db250cm9sbGVy%250AOjpGbGFzaDo6Rmxhc2hIYXNoewAGOgpAdXNlZHsA--f65405470eedc4a64 >>> defa69a0e78d22bd676cc0c; $Path=/; $Domain=.twitter.com". Illegal domain >>> attribut >>> e ".twitter.com". Domain of origin: "twitter.com" >>> >>> >>> >>> No feed updates in Esme. >>> >>> Vassil, would you want to fix this or shall I look into it. >>> >>> Imtiaz >>> >>> ----- Original Message ----- From: "Vassil Dichev" >>> To: >>> Sent: Friday, September 03, 2010 1:29 AM >>> Subject: Re: ESME-267 - Pooled links in popular links list >>> >>> >>>> Fixed. Now if you post the same link to a pool and to the public, you >>>> will notice that the href attribute points to the internal shortened >>>> URL in the former case and to the target URL in the latter case. This >>>> means that popularity statistics will only be gathered when links on >>>> public messages are clicked. >>>> >>>> An unique ID is still generated for all URLs but for links in pooled >>>> messages they're not visible. >>>> >>>> This should fix the problem. Does someone want to verify that we have >>>> indeed the correct behavior? >>>> >>>> Vassil >>>> >>>> >>>> On Wed, Sep 1, 2010 at 8:18 AM, Richard Hirsch >>>> wrote: >>>>> >>>>> Sounds like a god idea. >>>>> >>>>> D. >>>>> >>>>> On 8/31/10, Vassil Dichev wrote: >>>>>> >>>>>> Right, we just don't generate and store a unique ID for links in >>>>>> pools >>>>>> and will generate a different object on parsing. This way links which >>>>>> come from pools will point directly to the target URL and links from >>>>>> public messages will be redirected through the internal shortened >>>>>> URL, >>>>>> which will allow statistics to be collected. This won't break any >>>>>> functionality and I think it could be done fairly easily. >>>>>> >>>>>> I will assign ESME-267 to me if nobody objects to the proposed >>>>>> solution. >>>>>> >>>>>> Vassil >>>>>> >>>>>> >>>>>> On Tue, Aug 31, 2010 at 9:20 PM, Richard Hirsch >>>>>> >>>>>> wrote: >>>>>>> >>>>>>> Leave original link but just don't add it to PopularLinks. >>>>>>> >>>>>>> On 8/31/10, Ethan Jewett wrote: >>>>>>>> >>>>>>>> Oh, I see. Yes, that would make sense. So we would just leave the >>>>>>>> original link in there, right? >>>>>>>> >>>>>>>> Ethan >>>>>>>> >>>>>>>> On Tue, Aug 31, 2010 at 8:12 PM, Richard Hirsch >>>>>>>> >>>>>>>> wrote: >>>>>>>>> >>>>>>>>> I agree with the solution of just removing those links that >>>>>>>>> originate >>>>>>>>> in >>>>>>>>> pools. >>>>>>>>> >>>>>>>>> D. >>>>>>>>> >>>>>>>>> On 8/31/10, Vassil Dichev wrote: >>>>>>>>>> >>>>>>>>>> OK, I think this is a worse example, because there are many ways >>>>>>>>>> to >>>>>>>>>> find a list of URLs in a wiki (which were generally just not >>>>>>>>>> designed >>>>>>>>>> with privacy/security in mind). >>>>>>>>>> >>>>>>>>>> If you're willing to sacrifice convenience for security, the >>>>>>>>>> easiest >>>>>>>>>> change is not to parse URLs in messages in pools- it will appear >>>>>>>>>> as >>>>>>>>>> normal text, not as a hyperlink. The next thing we can do is set >>>>>>>>>> up >>>>>>>>>> a >>>>>>>>>> different type of URL which doesn't take you to the shortened >>>>>>>>>> URL, >>>>>>>>>> but >>>>>>>>>> directly to the target URL. >>>>>>>>>> >>>>>>>>>> If one really insists on shortening URLs in pools, then there >>>>>>>>>> must >>>>>>>>>> be >>>>>>>>>> one set of shortened URLs per pool. I don't think anyone will >>>>>>>>>> claim >>>>>>>>>> that this idea makes sense. >>>>>>>>>> >>>>>>>>>> Vassil >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Tue, Aug 31, 2010 at 11:35 AM, Ethan Jewett >>>>>>>>>> >>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>> I agree in theory with your assessment of the google docs >>>>>>>>>>> situation, >>>>>>>>>>> but I still think we're violating the expectation of security >>>>>>>>>>> around >>>>>>>>>>> pools. >>>>>>>>>>> >>>>>>>>>>> Take another example: An HR department is using a secure wiki to >>>>>>>>>>> discuss and organize an upcoming layoff. The wiki page is titled >>>>>>>>>>> "October layoff planning" and the URL is >>>>>>>>>>> https://hrwiki.corp.internal/October-layoff-planning. Someone >>>>>>>>>>> posts >>>>>>>>>>> this URL to the layoff-planning pool on esme (the same group of >>>>>>>>>>> people >>>>>>>>>>> with access to the wiki page) and a bunch of people in the pool >>>>>>>>>>> click >>>>>>>>>>> on it. Suddenly, the upcoming layoff has been announced to every >>>>>>>>>>> esme >>>>>>>>>>> user in the corporation. Whoops! >>>>>>>>>>> >>>>>>>>>>> The point is, maybe that private information shouldn't be in the >>>>>>>>>>> URL, >>>>>>>>>>> but a lot of applications do this whether or not it is a good >>>>>>>>>>> idea. >>>>>>>>>>> I >>>>>>>>>>> think we need to take that reality into account and change the >>>>>>>>>>> way >>>>>>>>>>> this works to avoid the possibility of these scenarios. >>>>>>>>>>> >>>>>>>>>>> Ethan >>>>>>>>>>> >>>>>>>>>>> On Tuesday, August 31, 2010, Vassil Dichev >>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>> Ethan, this defeats the purpose of having an URL shortener and >>>>>>>>>>>> it >>>>>>>>>>>> only >>>>>>>>>>>> gives you a false sense of security. Read my previous mail. >>>>>>>>>>>> >>>>>>>>>>>> Links have no notion of a pool. A link could come from messages >>>>>>>>>>>> in >>>>>>>>>>>> different pools or it might not be clicked "inside a message" >>>>>>>>>>>> at >>>>>>>>>>>> all. >>>>>>>>>>>> >>>>>>>>>>>> Let me know what you think. >>>>>>>>>>>> >>>>>>>>>>>> Vassil >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Tue, Aug 31, 2010 at 9:44 AM, Ethan Jewett >>>>>>>>>>>> >>>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> [Changed subject to start a new thread. Was: "New issues - a >>>>>>>>>>>>> couple >>>>>>>>>>>>> of >>>>>>>>>>>>> blockers for 1.1 release"] >>>>>>>>>>>>> >>>>>>>>>>>>> That's correct. The "Popular messages" functionality just >>>>>>>>>>>>> keeps a >>>>>>>>>>>>> counter of how many times a message has been resent. If you >>>>>>>>>>>>> look >>>>>>>>>>>>> at >>>>>>>>>>>>> the UserActor.scala, lines 197 & 198, you'll see that the >>>>>>>>>>>>> statistic >>>>>>>>>>>>> "ResendStat" is incremented when a message is resent, but only >>>>>>>>>>>>> if >>>>>>>>>>>>> the >>>>>>>>>>>>> message is not in a pool. Then when we want to find out what >>>>>>>>>>>>> the >>>>>>>>>>>>> most >>>>>>>>>>>>> popular messages are, we ask the PopStatsActor - for example >>>>>>>>>>>>> in >>>>>>>>>>>>> the >>>>>>>>>>>>> "popular" method of UserSnip.scala - line 213. >>>>>>>>>>>>> >>>>>>>>>>>>> On the other hand, the "LinkClicked is incremented in >>>>>>>>>>>>> UrlStore.scala >>>>>>>>>>>>> - >>>>>>>>>>>>> line 40. Here there is never a check to see if the link came >>>>>>>>>>>>> from >>>>>>>>>>>>> a >>>>>>>>>>>>> message in a pool. (This counter is used in the "links" method >>>>>>>>>>>>> in >>>>>>>>>>>>> UserSnip.scala, after the "popular" method.) >>>>>>>>>>>>> >>>>>>>>>>>>> I think we need to check if a link came from a pool before >>>>>>>>>>>>> incrementing the counter, but in order to do this we need to >>>>>>>>>>>>> record >>>>>>>>>>>>> what pool a link belonged to, so I think we need to make pool >>>>>>>>>>>>> part >>>>>>>>>>>>> of >>>>>>>>>>>>> the key of the UrlStore object and then populate this field >>>>>>>>>>>>> when >>>>>>>>>>>>> a >>>>>>>>>>>>> new >>>>>>>>>>>>> link is created. >>>>>>>>>>>>> >>>>>>>>>>>>> Ethan >>>>>>>>>>>>> >>>>>>>>>>>>> On Tue, Aug 31, 2010 at 8:11 AM, Imtiaz Ahmed H E >>>>>>>>>>>>> >>>>>>>>>>>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>> In the home when I type in a message sharing it with one pool >>>>>>>>>>>>>> and >>>>>>>>>>>>>> click >>>>>>>>>>>>>> resend it does not show up in Popular Messages. But if the >>>>>>>>>>>>>> message >>>>>>>>>>>>>> is >>>>>>>>>>>>>> public >>>>>>>>>>>>>> it shows up on resend in Popular Pessages. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Can you explain. Haven't gotten to Popular Links yet. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Imtiaz >>>>>>>>>>>>>> >>>>>>>>>>>>>> ----- Original Message ----- From: "Ethan Jewett" >>>>>>>>>>>>>> >>>>>>>>>>>>>> To: >>>>>>>>>>>>>> Sent: Tuesday, August 31, 2010 11:37 AM >>>>>>>>>>>>>> Subject: Re: New issues - a couple of blockers for 1.1 >>>>>>>>>>>>>> release >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>> >>>>>>>>>>>>>> The issue doesn't happen with Popular Messages, only with >>>>>>>>>>>>>> Popular >>>>>>>>>>>>>> Links. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I need to look into the implementation, but I have a feeling >>>>>>>>>>>>>> the >>>>>>>>>>>>>> Popular Links issue is going to be a headache. I believe that >>>>>>>>>>>>>> for a >>>>>>>>>>>>>> given link there is no way to tell what message it shows up >>>>>>>>>>>>>> in, >>>>>>>>>>>>>> which >>>>>>>>>>>>>> would make it impossible to tell if it is a link from a >>>>>>>>>>>>>> pooled >>>>>>>>>>>>>> message >>>>>>>>>>>>>> or not. We may have to modify the data model for storing >>>>>>>>>>>>>> links >>>>>>>>>>>>>> to >>>>>>>>>>>>>> flag >>>>>>>>>>>>>> the ones that started out in a pooled message... >>>>>>>>>>>>>> >>>>>>>>>>>>>> Regarding Pubsubhubbub, as Dick said, there's no hurry. I >>>>>>>>>>>>>> don't >>>>>>>>>>>>>> think >>>>>>>>>>>>>> I'll be working on it over the next couple of weeks. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks for all your efforts! >>>>>>>>>>>>>> >>>>>>>>>>>>>> Ethan >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Tue, Aug 31, 2010 at 4:20 AM, Imtiaz Ahmed H E >>>>>>>>>>>>>> >>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Re https://issues.apache.org/jira/browse/ESME-267 >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I haven't tried this but plan to fix it right away. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Tell me, is it only the links showing up in 'Popular Links' >>>>>>>>>>>>>>> or >>>>>>>>>>>>>>> is >>>>>>>>>>>>>>> that >>>>>>>>>>>>>>> a >>>>>>>>>>>>>>> problem with the message itself also showing up in >>>>>>>>>>>>>>> 'PopularMessages' >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Looks like I'll never get going with pubsubhubub ! First >>>>>>>>>>>>>>> there >>>>>>>>>>>>>>> was >>>>>>>>>>>>>>> Dick's >>>>>>>>>>>>>>> Release Planning mail with the pending 1.1 issues and now >>>>>>>>>>>>>>> here >>>>>>>>>>>>>>> are >>>>>>>>>>>>>>> some >>>>>>>>>>>>>>> more. Plan to get going after all 1.1 ending issues are >>>>>>>>>>>>>>> resolved. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> However, Ethan it was your issue originally and if you feel >>>>>>>>>>>>>>> you >>>>>>>>>>>>>>> want >>>>>>>>>>>>>>> to >>>>>>>>>>>>>>> take >>>>>>>>>>>>>>> it back again to push it to closure faster or something >>>>>>>>>>>>>>> please >>>>>>>>>>>>>>> do, >>>>>>>>>>>>>>> otherwise >>>>>>>>>>>>>>> I'll re-start on it once 1.1 is done... >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Imtiaz >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> ----- Original Message ----- From: "Richard Hirsch" >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> To: < >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>> >>> >> >