esme-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vassil Dichev <vdic...@apache.org>
Subject Re: Fw: ESME - 266 - my fix in progress, hopefully...
Date Fri, 03 Sep 2010 06:21:12 GMT
I don't know how Feed Reader does this, but the URL neither returns a
correct RSS feed, nor can I see any <link relative=.../> tag to
discover it. I don't think we should do anything non-standard here.

Vassil


On Fri, Sep 3, 2010 at 8:27 AM, Imtiaz Ahmed H E <in.imtiaz@gmail.com> wrote:
> Thanks Vassil, the feed url picked up from the browser toolbar went through;
>
> But do let me know if we should be able to pick up feeds from the RSS button
> on the nytimes home page at the bottom, just as any Feed Reader app
> apparently/presumably does.
>
> Imtiaz
>
> ----- Original Message ----- From: "Imtiaz Ahmed H E" <in.imtiaz@gmail.com>
> To: <esme-dev@incubator.apache.org>
> Sent: Friday, September 03, 2010 10:54 AM
> Subject: Re: Fw: ESME - 266 - my fix in progress, hopefully...
>
>
>> Okay, the Twitter feed went through when I started Esme a minute ago.
>>
>> Imtiaz
>>
>> ----- Original Message ----- From: "Imtiaz Ahmed H E"
>> <in.imtiaz@gmail.com>
>> To: <esme-dev@incubator.apache.org>
>> Sent: Friday, September 03, 2010 10:50 AM
>> Subject: Re: Fw: ESME - 266 - my fix in progress, hopefully...
>>
>>
>>> Sorry, I posted the mail in the wrong thread. I have now changed the
>>> subject line.
>>>
>>> My concern is, "feed link got thro' right-click on RSS icon at bottom of
>>> nytimes.com and "copy link location" in windows context menu and paste" into
>>> say, FeedReader (perhaps any other feed reader app too) sets up the feed
>>> correctly.
>>>
>>> So, should we handle this in Esme too... CAN WE ? Should I?
>>>
>>> By the way what about the Twitter problem in the mail I sent along with
>>> the nytimes proble (see below).
>>>
>>> Imtiaz
>>>
>>> ----- Original Message ----- From: "Vassil Dichev" <vdichev@apache.org>
>>> To: <esme-dev@incubator.apache.org>
>>> Sent: Friday, September 03, 2010 10:41 AM
>>> Subject: Re: ESME-267 - Pooled links in popular links list
>>>
>>>
>>> Well, it's obvious that if the XML couldn't parse the XML, then it's
>>> not valid XML. Add to this the fact that it contains a "meta" tag and
>>> it's clear that what's at the other end of the link was actually HTML.
>>> If you go to http://www.nytimes.com/rss, you'll notice this yourself.
>>>
>>> The real feed can be found by following the browser feed icon in the
>>> URL bar. It takes you to what the page describes as a feed in the tag
>>> <link rel="alternate" type="application/rss+xml" ... />. If you follow
>>> it, you will get to this URL:
>>>
>>> http://feeds.nytimes.com/nyt/rss/HomePage
>>>
>>> Try it, it should work.
>>>
>>> Vassil
>>>
>>>
>>> On Fri, Sep 3, 2010 at 2:35 AM, Imtiaz Ahmed H E <in.imtiaz@gmail.com>
>>> wrote:
>>>>
>>>> I have an action
>>>>
>>>> every 1 mins rss:http://www.nytimes.com/rss
>>>>
>>>> feed link got thro' right-click on RSS icon at bottom of nytimes.com and
>>>> "copy link location" in windows context menu and paste into esme with
>>>> rss:
>>>> prepended.
>>>>
>>>> and,
>>>>
>>>> every 1 mins
>>>> atom:http://twitter.com/statuses/user_timeline/esjewett.atom
>>>>
>>>> and I get in the Tomcat Window,
>>>>
>>>> WARN - Going to buffer response body of large or unknown size. Using
>>>> getResponse
>>>> BodyAsStream instead is recommended.
>>>> :37:31: expected closing tag of meta
>>>> <a
>>>> href="http://www.nytimes.com/pages/sports/index.html">Sports<
>>>> /a>
>>>> ^
>>>>
>>>>
>>>> and, apparently for the twitter feed,
>>>>
>>>> WARN - Going to buffer response body of large or unknown size. Using
>>>> getResponse
>>>> BodyAsStream instead is recommended.
>>>> :37:31: expected closing tag of meta
>>>> <a
>>>> href="http://www.nytimes.com/pages/sports/index.html">Sports<
>>>> /a>
>>>> ^
>>>> WARN - Cookie rejected: "$Version=0; k=122.167.31.233.1283470309076286;
>>>> $Path=/;
>>>> $Domain=.twitter.com". Illegal domain attribute ".twitter.com". Domain
>>>> of
>>>> origi
>>>> n: "twitter.com"
>>>> WARN - Cookie rejected: "$Version=0;
>>>> _twitter_sess=BAh7CDoPY3JlYXRlZF9hdGwrCNq2y
>>>>
>>>> tQqAToHaWQiJWRjMTNkMjQ2ZGRiYjA2%250AOTU1ZGZjMTc1NjMxMTZhN2I4IgpmbGFzaElDOidBY3Rp
>>>>
>>>> b25Db250cm9sbGVy%250AOjpGbGFzaDo6Rmxhc2hIYXNoewAGOgpAdXNlZHsA--f65405470eedc4a64
>>>> defa69a0e78d22bd676cc0c; $Path=/; $Domain=.twitter.com". Illegal domain
>>>> attribut
>>>> e ".twitter.com". Domain of origin: "twitter.com"
>>>>
>>>>
>>>>
>>>> No feed updates in Esme.
>>>>
>>>> Vassil, would you want to fix this or shall I look into it.
>>>>
>>>> Imtiaz
>>>>
>>>> ----- Original Message ----- From: "Vassil Dichev" <vdichev@apache.org>
>>>> To: <esme-dev@incubator.apache.org>
>>>> Sent: Friday, September 03, 2010 1:29 AM
>>>> Subject: Re: ESME-267 - Pooled links in popular links list
>>>>
>>>>
>>>>> Fixed. Now if you post the same link to a pool and to the public, you
>>>>> will notice that the href attribute points to the internal shortened
>>>>> URL in the former case and to the target URL in the latter case. This
>>>>> means that popularity statistics will only be gathered when links on
>>>>> public messages are clicked.
>>>>>
>>>>> An unique ID is still generated for all URLs but for links in pooled
>>>>> messages they're not visible.
>>>>>
>>>>> This should fix the problem. Does someone want to verify that we have
>>>>> indeed the correct behavior?
>>>>>
>>>>> Vassil
>>>>>
>>>>>
>>>>> On Wed, Sep 1, 2010 at 8:18 AM, Richard Hirsch <hirsch.dick@gmail.com>
>>>>> wrote:
>>>>>>
>>>>>> Sounds like a god idea.
>>>>>>
>>>>>> D.
>>>>>>
>>>>>> On 8/31/10, Vassil Dichev <vdichev@apache.org> wrote:
>>>>>>>
>>>>>>> Right, we just don't generate and store a unique ID for links
in
>>>>>>> pools
>>>>>>> and will generate a different object on parsing. This way links
which
>>>>>>> come from pools will point directly to the target URL and links
from
>>>>>>> public messages will be redirected through the internal shortened
>>>>>>> URL,
>>>>>>> which will allow statistics to be collected. This won't break
any
>>>>>>> functionality and I think it could be done fairly easily.
>>>>>>>
>>>>>>> I will assign ESME-267 to me if nobody objects to the proposed
>>>>>>> solution.
>>>>>>>
>>>>>>> Vassil
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Aug 31, 2010 at 9:20 PM, Richard Hirsch
>>>>>>> <hirsch.dick@gmail.com>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Leave original link but just don't add it to PopularLinks.
>>>>>>>>
>>>>>>>> On 8/31/10, Ethan Jewett <esjewett@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>> Oh, I see. Yes, that would make sense. So we would just
leave the
>>>>>>>>> original link in there, right?
>>>>>>>>>
>>>>>>>>> Ethan
>>>>>>>>>
>>>>>>>>> On Tue, Aug 31, 2010 at 8:12 PM, Richard Hirsch
>>>>>>>>> <hirsch.dick@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> I agree with the solution of just removing those
links that
>>>>>>>>>> originate
>>>>>>>>>> in
>>>>>>>>>> pools.
>>>>>>>>>>
>>>>>>>>>> D.
>>>>>>>>>>
>>>>>>>>>> On 8/31/10, Vassil Dichev <vdichev@apache.org>
wrote:
>>>>>>>>>>>
>>>>>>>>>>> OK, I think this is a worse example, because
there are many ways
>>>>>>>>>>> to
>>>>>>>>>>> find a list of URLs in a wiki (which were generally
just not
>>>>>>>>>>> designed
>>>>>>>>>>> with privacy/security in mind).
>>>>>>>>>>>
>>>>>>>>>>> If you're willing to sacrifice convenience for
security, the
>>>>>>>>>>> easiest
>>>>>>>>>>> change is not to parse URLs in messages in pools-
it will appear
>>>>>>>>>>> as
>>>>>>>>>>> normal text, not as a hyperlink. The next thing
we can do is set
>>>>>>>>>>> up
>>>>>>>>>>> a
>>>>>>>>>>> different type of URL which doesn't take you
to the shortened
>>>>>>>>>>> URL,
>>>>>>>>>>> but
>>>>>>>>>>> directly to the target URL.
>>>>>>>>>>>
>>>>>>>>>>> If one really insists on shortening URLs in pools,
then there
>>>>>>>>>>> must
>>>>>>>>>>> be
>>>>>>>>>>> one set of shortened URLs per pool. I don't think
anyone will
>>>>>>>>>>> claim
>>>>>>>>>>> that this idea makes sense.
>>>>>>>>>>>
>>>>>>>>>>> Vassil
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Aug 31, 2010 at 11:35 AM, Ethan Jewett
>>>>>>>>>>> <esjewett@gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> I agree in theory with your assessment of
the google docs
>>>>>>>>>>>> situation,
>>>>>>>>>>>> but I still think we're violating the expectation
of security
>>>>>>>>>>>> around
>>>>>>>>>>>> pools.
>>>>>>>>>>>>
>>>>>>>>>>>> Take another example: An HR department is
using a secure wiki to
>>>>>>>>>>>> discuss and organize an upcoming layoff.
The wiki page is titled
>>>>>>>>>>>> "October layoff planning" and the URL is
>>>>>>>>>>>> https://hrwiki.corp.internal/October-layoff-planning.
Someone
>>>>>>>>>>>> posts
>>>>>>>>>>>> this URL to the layoff-planning pool on esme
(the same group of
>>>>>>>>>>>> people
>>>>>>>>>>>> with access to the wiki page) and a bunch
of people in the pool
>>>>>>>>>>>> click
>>>>>>>>>>>> on it. Suddenly, the upcoming layoff has
been announced to every
>>>>>>>>>>>> esme
>>>>>>>>>>>> user in the corporation. Whoops!
>>>>>>>>>>>>
>>>>>>>>>>>> The point is, maybe that private information
shouldn't be in the
>>>>>>>>>>>> URL,
>>>>>>>>>>>> but a lot of applications do this whether
or not it is a good
>>>>>>>>>>>> idea.
>>>>>>>>>>>> I
>>>>>>>>>>>> think we need to take that reality into account
and change the
>>>>>>>>>>>> way
>>>>>>>>>>>> this works to avoid the possibility of these
scenarios.
>>>>>>>>>>>>
>>>>>>>>>>>> Ethan
>>>>>>>>>>>>
>>>>>>>>>>>> On Tuesday, August 31, 2010, Vassil Dichev
<vdichev@apache.org>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Ethan, this defeats the purpose of having
an URL shortener and
>>>>>>>>>>>>> it
>>>>>>>>>>>>> only
>>>>>>>>>>>>> gives you a false sense of security.
Read my previous mail.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Links have no notion of a pool. A link
could come from messages
>>>>>>>>>>>>> in
>>>>>>>>>>>>> different pools or it might not be clicked
"inside a message"
>>>>>>>>>>>>> at
>>>>>>>>>>>>> all.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Let me know what you think.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Vassil
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Tue, Aug 31, 2010 at 9:44 AM, Ethan
Jewett
>>>>>>>>>>>>> <esjewett@gmail.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> [Changed subject to start a new thread.
Was: "New issues - a
>>>>>>>>>>>>>> couple
>>>>>>>>>>>>>> of
>>>>>>>>>>>>>> blockers for 1.1 release"]
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> That's correct. The "Popular messages"
functionality just
>>>>>>>>>>>>>> keeps a
>>>>>>>>>>>>>> counter of how many times a message
has been resent. If you
>>>>>>>>>>>>>> look
>>>>>>>>>>>>>> at
>>>>>>>>>>>>>> the UserActor.scala, lines 197 &
198, you'll see that the
>>>>>>>>>>>>>> statistic
>>>>>>>>>>>>>> "ResendStat" is incremented when
a message is resent, but only
>>>>>>>>>>>>>> if
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>> message is not in a pool. Then when
we want to find out what
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>> most
>>>>>>>>>>>>>> popular messages are, we ask the
PopStatsActor - for example
>>>>>>>>>>>>>> in
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>> "popular" method of UserSnip.scala
- line 213.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On the other hand, the "LinkClicked
is incremented in
>>>>>>>>>>>>>> UrlStore.scala
>>>>>>>>>>>>>> -
>>>>>>>>>>>>>> line 40. Here there is never a check
to see if the link came
>>>>>>>>>>>>>> from
>>>>>>>>>>>>>> a
>>>>>>>>>>>>>> message in a pool. (This counter
is used in the "links" method
>>>>>>>>>>>>>> in
>>>>>>>>>>>>>> UserSnip.scala, after the "popular"
method.)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I think we need to check if a link
came from a pool before
>>>>>>>>>>>>>> incrementing the counter, but in
order to do this we need to
>>>>>>>>>>>>>> record
>>>>>>>>>>>>>> what pool a link belonged to, so
I think we need to make pool
>>>>>>>>>>>>>> part
>>>>>>>>>>>>>> of
>>>>>>>>>>>>>> the key of the UrlStore object and
then populate this field
>>>>>>>>>>>>>> when
>>>>>>>>>>>>>> a
>>>>>>>>>>>>>> new
>>>>>>>>>>>>>> link is created.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Ethan
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Tue, Aug 31, 2010 at 8:11 AM,
Imtiaz Ahmed H E
>>>>>>>>>>>>>> <in.imtiaz@gmail.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> In the home when I type in a
message sharing it with one pool
>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>> click
>>>>>>>>>>>>>>> resend it does not show up in
Popular Messages. But if the
>>>>>>>>>>>>>>> message
>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>> public
>>>>>>>>>>>>>>> it shows up on resend in Popular
Pessages.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Can you explain. Haven't gotten
to Popular Links yet.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Imtiaz
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> ----- Original Message -----
From: "Ethan Jewett"
>>>>>>>>>>>>>>> <esjewett@gmail.com>
>>>>>>>>>>>>>>> To: <esme-dev@incubator.apache.org>
>>>>>>>>>>>>>>> Sent: Tuesday, August 31, 2010
11:37 AM
>>>>>>>>>>>>>>> Subject: Re: New issues - a couple
of blockers for 1.1
>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> The issue doesn't happen with
Popular Messages, only with
>>>>>>>>>>>>>>> Popular
>>>>>>>>>>>>>>> Links.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I need to look into the implementation,
but I have a feeling
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> Popular Links issue is going
to be a headache. I believe that
>>>>>>>>>>>>>>> for a
>>>>>>>>>>>>>>> given link there is no way to
tell what message it shows up
>>>>>>>>>>>>>>> in,
>>>>>>>>>>>>>>> which
>>>>>>>>>>>>>>> would make it impossible to tell
if it is a link from a
>>>>>>>>>>>>>>> pooled
>>>>>>>>>>>>>>> message
>>>>>>>>>>>>>>> or not. We may have to modify
the data model for storing
>>>>>>>>>>>>>>> links
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>> flag
>>>>>>>>>>>>>>> the ones that started out in
a pooled message...
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Regarding Pubsubhubbub, as Dick
said, there's no hurry. I
>>>>>>>>>>>>>>> don't
>>>>>>>>>>>>>>> think
>>>>>>>>>>>>>>> I'll be working on it over the
next couple of weeks.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks for all your efforts!
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Ethan
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Tue, Aug 31, 2010 at 4:20
AM, Imtiaz Ahmed H E
>>>>>>>>>>>>>>> <in.imtiaz@gmail.com>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Re https://issues.apache.org/jira/browse/ESME-267
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I haven't tried this but
plan to fix it right away.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Tell me, is it only the links
showing up in 'Popular Links'
>>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>> problem with the message
itself also showing up in
>>>>>>>>>>>>>>>> 'PopularMessages'
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Looks like I'll never get
going with pubsubhubub ! First
>>>>>>>>>>>>>>>> there
>>>>>>>>>>>>>>>> was
>>>>>>>>>>>>>>>> Dick's
>>>>>>>>>>>>>>>> Release Planning mail with
the pending 1.1 issues and now
>>>>>>>>>>>>>>>> here
>>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>> some
>>>>>>>>>>>>>>>> more. Plan to get going after
all 1.1 ending issues are
>>>>>>>>>>>>>>>> resolved.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> However, Ethan it was your
issue originally and if you feel
>>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>> want
>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>> take
>>>>>>>>>>>>>>>> it back again to push it
to closure faster or something
>>>>>>>>>>>>>>>> please
>>>>>>>>>>>>>>>> do,
>>>>>>>>>>>>>>>> otherwise
>>>>>>>>>>>>>>>> I'll re-start on it once
1.1 is done...
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Imtiaz
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> ----- Original Message -----
From: "Richard Hirsch"
>>>>>>>>>>>>>>>> <hirsch.dick@gmail.com>
>>>>>>>>>>>>>>>> To: <
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>
>>>>
>>>
>>
>
>

Mime
View raw message