cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stu Hood (JIRA)" <j...@apache.org>
Subject [jira] Updated: (CASSANDRA-579) Add support to io.Streaming API for sending Streams
Date Sat, 27 Feb 2010 21:17:10 GMT

     [ https://issues.apache.org/jira/browse/CASSANDRA-579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Stu Hood updated CASSANDRA-579:
-------------------------------

    Description: 
The io.Streaming API currently requires a file on disk to stream, which means that bootstrap
and repairs need to perform an anti-compaction that writes a bunch of data to disk, only to
have it be deleted after the streaming has finished.

EDIT: Deleted reference to using streaming as a client API: it wouldn't provide enough benefit
over using the BMT interface, due to fragility.

  was:
The io.Streaming API currently requires a file on disk to stream, which means that bootstrap
and repairs need to perform an anti-compaction that writes a bunch of data to disk, only to
have it be deleted after the streaming has finished.

Ideally, the Streaming API should allow for streaming from an InputStream (or any other class
we think we need to design to make the streaming as efficient as possible). That way, anti-compaction
for repair/bootstrap does not perform any writing: it simply streams the relevant portion
of the file to the neighbor.

Additionally, this opens up interesting possibilities, such as providing the Streaming API
as a (Java only?) client API. One use case would be for a Hadoop OutputFormat: rather than
writing BinaryMemtables, the OutputFormat could literally write an SSTable to the stream.
This might require better integration with gossip, to ensure that you aren't writing to the
completely wrong node.


> Add support to io.Streaming API for sending Streams
> ---------------------------------------------------
>
>                 Key: CASSANDRA-579
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-579
>             Project: Cassandra
>          Issue Type: Improvement
>            Reporter: Stu Hood
>             Fix For: 0.7
>
>
> The io.Streaming API currently requires a file on disk to stream, which means that bootstrap
and repairs need to perform an anti-compaction that writes a bunch of data to disk, only to
have it be deleted after the streaming has finished.
> EDIT: Deleted reference to using streaming as a client API: it wouldn't provide enough
benefit over using the BMT interface, due to fragility.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message