cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sylvain Lebresne (JIRA)" <>
Subject [jira] [Commented] (CASSANDRA-10990) Support streaming of older version sstables in 3.0
Date Thu, 21 Jan 2016 14:55:39 GMT


Sylvain Lebresne commented on CASSANDRA-10990:

The problem will be compact tables. With them, old nodes have their cells for declared columns
intermingled with other cells and we need to get all the former ones first for the new format
(as they become static columns). When reading old sstables, we end up having to sometimes
double read partitions during compaction (thankfully it's only for some tables and it's a
one time thing) and do some tricks for reads (which are too long to explain here but one can
look at {{ThriftResultsMerger}} if interested).

Anyway, we can't easily do this for streaming. The only viable solution I can see would be
to first write all the "dynamic" cells to some temporary location collecting the "static"
ones while doing it, then writing the static ones and then copying back the dynamic ones.
Doable, though it's getting complicated.

I also want to note that streaming sstables is in itself useless. What we care is the features
that use streaming, i.e. bootstrap, moving and repair. And at least for repair, this won't
work out of the box because 3.0 don't compute the same digest than 2.X nodes. Now, 3.0 nodes
are actually able to compute the 2.X ones (which we use for digest queries), but we need to
add the code so they pick the right version to compute. As for bootstrap/moving, they "should"
work out of the box in mixed cluster (providing streaming works) but we've changed enough
in 3.0, including with schemas, that we need a good battery of tests before claiming it does.

So anyway, we can likely solve all those problems, but it's worth noting that this ticket
is probably a rather big one if you include the testing necessary (and I think it's better
to say "sorry but you can't do streaming based operations during upgrade" than pretending
it works but having it break easily in practice).

> Support streaming of older version sstables in 3.0
> --------------------------------------------------
>                 Key: CASSANDRA-10990
>                 URL:
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Streaming and Messaging
>            Reporter: Jeremy Hanna
>            Assignee: Paulo Motta
> In 2.0 we introduced support for streaming older versioned sstables (CASSANDRA-5772).
 In 3.0, because of the rewrite of the storage layer, this became no longer supported.  So
currently, while 3.0 can read sstables in the 2.1/2.2 format, it cannot stream the older versioned
sstables.  We should do some work to make this still possible to be consistent with what CASSANDRA-5772

This message was sent by Atlassian JIRA

View raw message