cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Peter Schuller (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-3494) Streaming is mono-threaded (the bulk loader too by extension)
Date Wed, 28 Dec 2011 19:20:31 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-3494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13176775#comment-13176775
] 

Peter Schuller commented on CASSANDRA-3494:
-------------------------------------------

I did test it, yes. Streaming does work, and it does indeed not retain an idle thread per
destination beyond end of streaming and timeout (verified with jstack). I'd have to go back
to the code again (I also looked at ThreadPoolExecutor) to address your concern specifically.
What problem are you seeing?

                
> Streaming is mono-threaded (the bulk loader too by extension)
> -------------------------------------------------------------
>
>                 Key: CASSANDRA-3494
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-3494
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>    Affects Versions: 0.8.0
>            Reporter: Sylvain Lebresne
>            Assignee: Peter Schuller
>            Priority: Minor
>             Fix For: 1.1
>
>         Attachments: CASSANDRA-3494-0.8-prelim.txt, CASSANDRA-3494-1.0.txt
>
>
> The streamExecutor is define as:
> {noformat}
> streamExecutor_ = new DebuggableThreadPoolExecutor("Streaming", Thread.MIN_PRIORITY);
> {noformat}
> In the meantime, in DebuggableThreadPoolExecutor.java:
> {noformat}
> public DebuggableThreadPoolExecutor(String threadPoolName, int priority)
> {
>    this(1, Integer.MAX_VALUE, TimeUnit.SECONDS, new LinkedBlockingQueue<Runnable>(),
new NamedThreadFactory(threadPoolName, priority));
> }
> {noformat}
> In other word, since the core pool size is 1 and the queue unbounded, tasks will always
queued and the executor is essentially mono-threaded.
> This is clearly not necessary since we already have stream throttling nowadays. And it
could be a limiting factor in the case of the bulk loader.
> Besides, I would venture that this maybe was not the intention, because putting the max
core size to MAX_VALUE would suggest that the intention was to spawn threads on demand. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message