cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Peter Schuller (Resolved) (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (CASSANDRA-3549) streaming is hard-coded to be single-threaded
Date Thu, 01 Dec 2011 11:58:39 GMT

     [ https://issues.apache.org/jira/browse/CASSANDRA-3549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Peter Schuller resolved CASSANDRA-3549.
---------------------------------------

    Resolution: Duplicate
    
> streaming is hard-coded to be single-threaded
> ---------------------------------------------
>
>                 Key: CASSANDRA-3549
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-3549
>             Project: Cassandra
>          Issue Type: Bug
>            Reporter: Peter Schuller
>            Assignee: Peter Schuller
>            Priority: Minor
>
> It turns out, after we were scratching our heads as to why we weren't streaming as fast
as expected when we bootstrapped 24 additional nodes in a 24 node cluster, that a node will
only ever stream one outgoing file at a time. This is because the MessagingService instantiates
it's streamExecturor_ in such a way that it gets a maximum size of 1.
> This is particularly bad because there is no mechanism to ensure that destination nodes
stream "evenly" from availabe source nodes, so some source nodes were dogpiled.
> Meanwhile, the streams that were active were bottlenecking (in our case) on the destination
side due to writes. In the end the aggregate throughput bootstrapping in the cluster was far
below intended.
> Will submit a patch once we've confirmed it seems safe and that there's no concurrency
issue lurking.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message