cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sylvain Lebresne (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-3668) Performance of sstableloader is affected in 1.0.x
Date Wed, 02 May 2012 13:32:50 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-3668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13266560#comment-13266560
] 

Sylvain Lebresne commented on CASSANDRA-3668:
---------------------------------------------

bq.  I just felt weird to see the thread name of IncomingTCPConn as default (like 'Thread-10')
on the other hand OutboundTCPConn gets name like 'WRITE-192.168.1.141'

I agree, and I was good with
{noformat}
super("READ-" + socket.getRemoteSocketAddress());
{noformat}
My "problem" was with
{noformat}
setName("streaming from " + socket.getRemoteSocketAddress());
{noformat}

bq. I also changed the name of variable streamingThreadsPerNode to streamExecutorCoreSize
and set its default value to 0, in order to preserve current behavior.

In fact my argument was that using 0 is bad but that was argumented already on the original
issue (https://issues.apache.org/jira/browse/CASSANDRA-3494?focusedCommentId=13178349&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13178349)
but somehow I forgot to commit my 'cleanup' patch. Anyway, I took the liberty to commit said
cleanup patch (so you will need to rebase this one if you don't mind).

bq. I modified to have per session retries, since we are closing session after retries exceed
the limit.

Yeah, I suppose the best way to handle that would be a retry-per-file to avoid failing a seesion
that may make progress on some files, but that's a bigger refactor than is worth here.


A few additional remarks:
* StreamInSession.isFinished handling is not thread safe, we should use an AtomicBoolean and
test-an-set it. However AbstractStreamSession.close() already use the same trick to ensure
closeInternal() is called only once, so maybe we could just move the code to add the sstables
and secondary indexes to closeInternal. 
* StreamInSession.setCurrentFile() is a no-op and should be removed.

                
> Performance of sstableloader is affected in 1.0.x
> -------------------------------------------------
>
>                 Key: CASSANDRA-3668
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-3668
>             Project: Cassandra
>          Issue Type: Bug
>          Components: API
>    Affects Versions: 1.0.0
>            Reporter: Manish Zope
>            Assignee: Yuki Morishita
>            Priority: Minor
>              Labels: streaming
>             Fix For: 1.1.1
>
>         Attachments: 3668-1.1.txt, 3688-reply_before_closing_writer.txt, sstable-loader
performance.txt
>
>   Original Estimate: 48h
>  Remaining Estimate: 48h
>
> One of my colleague had reported the bug regarding the degraded performance of the sstable
generator and sstable loader.
> ISSUE :- https://issues.apache.org/jira/browse/CASSANDRA-3589 
> As stated in above issue generator performance is rectified but performance of the sstableloader
is still an issue.
> 3589 is marked as duplicate of 3618.Both issues shows resolved status.But the problem
with sstableloader still exists.
> So opening other issue so that sstbleloader problem should not go unnoticed.
> FYI : We have tested the generator part with the patch given in 3589.Its Working fine.
> Please let us know if you guys require further inputs from our side.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message