cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nirmal Ranganathan (JIRA)" <>
Subject [jira] Updated: (CASSANDRA-1189) Refactor streaming
Date Tue, 27 Jul 2010 23:59:20 GMT


Nirmal Ranganathan updated CASSANDRA-1189:

    Attachment: 0001-Refactored-streaming-to-make-it-more-streamlined.patch

There's a lot of changes in this one. I'll try to explain some of the changes here.

- Anything to do with StreamInitiate is removed and some of what it does is incorporated in
StreamIn and StreamOut. 
- StreamContext is added which is InetAddress+sessionId, which acts as the key for all transactions.

- StreamHeader is added, which contains info on each stream as opposed to doing all the initiate

- Source maintains the list of files to stream in StreamOutManager. There's one per context.
- Destination doesn't maintain anything except for when the stream is active via StreamInManager.activeStreams.
- Destination sends a FileStatus message for the received stream

- Destination initiates a request context in StreamInManager.
- Source compiles the list of files and adds it to a StreamOutManager, this is just for book-keeping
purposes and troubleshooting.
- Source streams first file with header containing info of all the other files.
- Destination maintains the list of files in StreamInManager and takes control from this point.
- Destination requests the next file until done and re-requests on error

> Refactor streaming
> ------------------
>                 Key: CASSANDRA-1189
>                 URL:
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>    Affects Versions: 0.7 beta 1
>            Reporter: Gary Dusbabek
>            Assignee: Nirmal Ranganathan
>            Priority: Critical
>             Fix For: 0.7.0
>         Attachments: 0001-Refactored-streaming-to-make-it-more-streamlined.patch
> The current architecture is buggy because it makes the assumption that only one stream
can be in process between two nodes at a given time, and stream send order never changes.
 Because of this, the ACK process gets fouled up when other services wish to stream files.
> The process is somewhat contorted too (request, initiate, initiate done, send).

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message