streams-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (STREAMS-311) TwitterUserInformationProvider stalls with > 20 items provided
Date Thu, 14 May 2015 22:56:00 GMT

    [ https://issues.apache.org/jira/browse/STREAMS-311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14544543#comment-14544543
] 

ASF GitHub Bot commented on STREAMS-311:
----------------------------------------

Github user jfrazee commented on a diff in the pull request:

    https://github.com/apache/incubator-streams/pull/217#discussion_r30371934
  
    --- Diff: streams-contrib/streams-provider-twitter/src/main/java/org/apache/streams/twitter/provider/TwitterFollowingProviderTask.java
---
    @@ -141,7 +141,7 @@ else if( endpoint.equals("friends") )
                 catch(Exception e) {
                     keepTrying += TwitterErrorHandler.handleTwitterError(client, e);
                 }
    -        } while (curser != 0 && keepTrying < 10);
    +        } while (curser != 0 && keepTrying < 50);
    --- End diff --
    
    I think this should be left out of this PR in favor of making the TwitterErrorHandler
respect the Twitter4j RateLimitStatus reset time. I'm volunteering to do that, so unless STREAMS-311
can't be fixed without it I'd say to put this back to 10.


> TwitterUserInformationProvider stalls with > 20 items provided
> --------------------------------------------------------------
>
>                 Key: STREAMS-311
>                 URL: https://issues.apache.org/jira/browse/STREAMS-311
>             Project: Streams
>          Issue Type: Bug
>            Reporter: Steve Blackmon
>
> The way parallelism is handled in TwitterUserInformationProvider can cause the provider
to stall when more threads are created than the size of the execution service queue which
is hard-coded to twenty.
> The size of the execution service queue should equal the number of threads to ensure
that startStream() does not block trying to add more threads, because until startStream()
completes the local runtime does not poll readCurrent().



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message