cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Paulo Motta (JIRA)" <>
Subject [jira] [Commented] (CASSANDRA-10990) Support streaming of older version sstables in 3.0
Date Fri, 22 Jan 2016 18:34:39 GMT


Paulo Motta commented on CASSANDRA-10990:

Many thanks for the detailed explanation and clarification [~slebresne]!

I believe the purpose of this ticket is more to allow streaming of old sstables to 3.0 vnodes
via {{sstableloader}} as well as support bootstrap and move on 3.0 nodes that did not complete
{{upgradesstables}} (dense nodes) than to actually provide full 2.x-3.0 streaming compatibility
(we can leave this to CASSANDRA-8110). There seems to be additional complications with repair
due to different digest formats, so we can address that in a separate ticket if necessary.

I was able to achieve streaming of pre-3.0 sstables fairly transparently with the addition
of a {{RewindableDataInputStreamPlus}} wrapper input stream that allows rewinding a source
stream, so we can leverage the exactly same code path of reading pre-3.0 sstables ({{OldFormatIterator}})
to do the static compact table handling. Currently, the {{RewindableDataInputStreamPlus}}
works in memory, so next step is to spill the buffer to disk if its sizes goes above a certain

I agree we need to do extensive testing before calling this supported. I propose to extend
CASSANDRA-10563 upgrade dtests to perform a bootstrap of a new node, without performing {{upgradesstables}}
on upgraded nodes (so they will contain only old sstables), and check that all streamed data
will be readable via thrift and/or cql.

I updated the previous branch with the latest changes.

> Support streaming of older version sstables in 3.0
> --------------------------------------------------
>                 Key: CASSANDRA-10990
>                 URL:
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Streaming and Messaging
>            Reporter: Jeremy Hanna
>            Assignee: Paulo Motta
> In 2.0 we introduced support for streaming older versioned sstables (CASSANDRA-5772).
 In 3.0, because of the rewrite of the storage layer, this became no longer supported.  So
currently, while 3.0 can read sstables in the 2.1/2.2 format, it cannot stream the older versioned
sstables.  We should do some work to make this still possible to be consistent with what CASSANDRA-5772

This message was sent by Atlassian JIRA

View raw message