cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jun Rao (JIRA)" <>
Subject [jira] Updated: (CASSANDRA-220) TCP writes get stuck
Date Mon, 08 Jun 2009 04:24:07 GMT


Jun Rao updated CASSANDRA-220:

    Attachment: issue220.patchv1

Attache a patch to trunk by expanding the synchronization scope in TcpConnection.doPendingWrites().

We need to patch to 0.3 branch too.

Also, we need to revisit the code in TcpConnection. Under heavy writes, is it better to keep
accumulating writes until we run out of memory, or to put a cap on the write buffer and simply
block and possibly timeout the writer.

> TCP writes get stuck
> --------------------
>                 Key: CASSANDRA-220
>                 URL:
>             Project: Cassandra
>          Issue Type: Bug
>    Affects Versions: trunk
>            Reporter: Jun Rao
>            Assignee: Jun Rao
>             Fix For: trunk
>         Attachments: issue220.patchv1
> In our test cluster, I observed that on some nodes, TCP writes get accumulated in TcpConnection.pendingWrites.
However, the selector never gets a ready to write event. As a result, all writes get stuck.
This is because write(message) and doPendingWrites() are not fully synchronized.  This following
situation can happen:
> 1. write(message) adds stuff to pendingWrites.
> 2. a ready to write event happens; in doPendingWrites() buffered requests are written
to socket
> 3. another write request happens, in write(message), the test for pendingWrites.isEmpty()
is false
> 4. doPendingWrites() finishes writing all buffered request to socket
> 5. in write(message), the new request is added to pendingWrites
> Now, ready to write events will never happen again and all write requests get stuck in

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message