cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yuki Morishita (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-10990) Support streaming of older version sstables in 3.0
Date Thu, 25 Feb 2016 21:59:18 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-10990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15167951#comment-15167951
] 

Yuki Morishita commented on CASSANDRA-10990:
--------------------------------------------

Thanks for the update. Here are my review comments.

* StreamDeserializer determines tmp directory to where disk can be hold 2 * the size of streaming
SSTable, but isn't it too much? SSTable can be few hundred GB. You need to cap that.
* Speaking of capping the size of tmp buffer file, we may want to respect the parameter of
{{InputStream#mark(int readLimit)}}. We just set read limit to 
2.147GB and make it the limitation of this functionality. Users can always upgradesstable
if needed.
* RewindableInputStream - I don't think we need {{SyncUtil.sync}} on buffer file.
* RewindableInputStream only deletes buffer file at close, but it is not closed when stream
finishes. You cannot close with {{RewindableInputStream#close}} because that will cause underlying
socket to be closed.
* RewindableDataInputStreamPlus - {{bytesPastMark}} returns underlying stream's {{available()}},
but I don't think that is suitable there. {{available}} (by contract) just returns estimated
bytes to be read without blocking. You should return {{pos}} from {{RewindableInputStream}}.
That said, I think it is better to integrate {{RewindableDataInputStreamPlus}} and {{RewindableInputStream}}.
* Initial capacity seems to be set as 128 bytes, not 128 Kbytes. Also I prefer passing this
value to constructor as well so you don't have to deal with {{System.setProperty}} in unit
test.


> Support streaming of older version sstables in 3.0
> --------------------------------------------------
>
>                 Key: CASSANDRA-10990
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-10990
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Streaming and Messaging
>            Reporter: Jeremy Hanna
>            Assignee: Paulo Motta
>
> In 2.0 we introduced support for streaming older versioned sstables (CASSANDRA-5772).
 In 3.0, because of the rewrite of the storage layer, this became no longer supported.  So
currently, while 3.0 can read sstables in the 2.1/2.2 format, it cannot stream the older versioned
sstables.  We should do some work to make this still possible to be consistent with what CASSANDRA-5772
provided.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message