camel-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Claus Ibsen (JIRA)" <j...@apache.org>
Subject [jira] Commented: (CAMEL-3124) polling of feeds in FeedEntryPollingConsumer is broken.
Date Fri, 17 Sep 2010 11:28:40 GMT

    [ https://issues.apache.org/activemq/browse/CAMEL-3124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=61893#action_61893
] 

Claus Ibsen commented on CAMEL-3124:
------------------------------------

And you can set {{throttleEntries=false}} to let it parse the entire feed at once, and hence
it doesn't keep state in memory between polls.

> polling of feeds in FeedEntryPollingConsumer is broken.
> -------------------------------------------------------
>
>                 Key: CAMEL-3124
>                 URL: https://issues.apache.org/activemq/browse/CAMEL-3124
>             Project: Apache Camel
>          Issue Type: Bug
>          Components: camel-rss
>    Affects Versions: 2.4.0
>         Environment: ubuntu 10.4
> java 6
>            Reporter: Ernst Bunders
>            Assignee: Claus Ibsen
>            Priority: Minor
>             Fix For: 2.5.0
>
>
> The FeedEntryPollingConsumer class implements the poll() method for the 'splitEntries'
mode of the RssEndpoint is broken.
> You can think of two ways that polling feeds could work:
> 1) A feed is created, then one item is processed, then the delay, then process another
item. This way the feed is kept between calls to poll().
> 2) A feed is created, then all the items are processed, the feed is cleared, and then
the delay.
> But the way it presently works:
> A feed is created, one items is processed, and the feed is cleared, then the delay, and
again the feed is created and the next item is cleared.
> This is clearly wrong. Feed entries can be missed, because the index of the next item
to process is stored over polls but the list isn't. Also this creates a big network overhead
when polling very active feeds such as twitter search...
> This is easy to fix. In the below code:
> {code}
> public void poll() throws Exception {
>         Object feed = createFeed();
>         populateList(feed);   
>         while (hasNextEntry()) {
>             Object entry = list.get(entryIndex--);
>             boolean valid = true;
>             if (entryFilter != null) {
>                 valid = entryFilter.isValidEntry(endpoint, feed, entry);
>             }
>             if (valid) {
>                 Exchange exchange = endpoint.createExchange(feed, entry);
>                 getProcessor().process(exchange);
>                 // return and wait for the next poll to continue from last time (this
consumer is stateful)
>                 return;
>             }
>         }
> {code}
> The return (at line 56 of org.apache.camel.component.feed.FeedEntryPollingConsumer) should
be deleted.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message