cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stefania (JIRA)" <>
Subject [jira] [Commented] (CASSANDRA-11521) Implement streaming for bulk read requests
Date Fri, 20 May 2016 08:28:13 GMT


Stefania commented on CASSANDRA-11521:

Thank you for your input [~snazy] and [~benedict].

bq. However these streams can be arbitrarily large, so certainly we don't want to evaluate
the entire query to permit releasing the sstables.

Noted, I will keep only a small amount of buffers in memory. I kind of came to this conclusion
after reading Robert's comment, the time-bound lifespan is an excellent idea, thank you for
suggesting it.

bq. Note, that the OpOrder should not be used by these queries - actual references should
be taken so that long lifespans have no impact.

You mean actual references to the sstables via something like CFS.selectAndReference? I don't
understand how we can read from off-heap memtables without using OpOrder since their memory
would be invalidated once they are flushed. Above I mentioned releasing sstables but I actually
meant both sstables and memtables, do you think it's a bad idea to block memtable flushing?

bq. The code that takes these references really needs to be fixed, also, so that the races
to update the data tracker don't cause temporary "infinite" loops - like we see for range
queries today.

Sorry but I really cannot understand how PartitionRangeReadCommand.queryStorage may cause
races when updating the data tracker view atomic reference (I presume this is what you meant).

> Implement streaming for bulk read requests
> ------------------------------------------
>                 Key: CASSANDRA-11521
>                 URL:
>             Project: Cassandra
>          Issue Type: Sub-task
>          Components: Local Write-Read Paths
>            Reporter: Stefania
>            Assignee: Stefania
>             Fix For: 3.x
> Allow clients to stream data from a C* host, bypassing the coordination layer and eliminating
the need to query individual pages one by one.

This message was sent by Atlassian JIRA

View raw message