cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthew F. Dennis (JIRA)" <>
Subject [jira] [Updated] (CASSANDRA-1278) Make bulk loading into Cassandra less crappy, more pluggable
Date Fri, 29 Apr 2011 21:57:03 GMT


Matthew F. Dennis updated CASSANDRA-1278:

    Attachment: 1278-cassandra-0.7-v2.txt

{quote}Did you accidentally remove the buffering for non-streaming connections in IncomingTCPConnection?{quote}
It was intentional as previously only the streaming was buffered (at 4k) but the bulk of the
work uses the socket channel; only the size and header are read from input and the header
uses readFully.  It adds an extra call when constructing the stream because of the size but
avoids copying the data into the buffer (in the BufferedStream) and then into the byte array.
 We could lower some of those calls by reading both the magic and the header int at the same
time into a ByteBuffer and then viewing it as an IntBuffer but I don't think that buys you
anything as it only happens on a new connection.  It also avoids bugs where something has
been read from the socket into the buffer and then the socket channel is used later even though
the buffer may not have been fully drained.

{quote}What kind of failures are supported during a load?{quote}

On the server side all failures result in the same behaviour: close socket, delete temp files.
On the client side if flushing of a BareMemtable to the server fails the proxy will log it
and continue running.

In both cases any data that was being loaded via the proxy needs to be reloaded.

{quote}What's the proper behaviour for flush_proxy when some hosts fail?{quote}
log failed flushes and continue running; any data that was being loaded via the proxy needs
to be reloaded.

{quote}Could we avoid coding in knowledge of the file format in the inner loop of IncomingLoaderStreamReader?
I would much, much prefer that non-file-format-specific framing be added, and it would have
the added benefit of not requiring as many system calls (4 per row vs 2 per frame){quote}

We could construct something that buffers up X amount of data and then frames the data being
sent and change the inner loop to decompose that but it's extra complexity, code and overhead.
 If we buffer it on the other side we consume more memory for a longer period of time (thus
giving it a better chance that it needs to promoted and/or compacted)  adding to the already
problematic GC pressure.  If we don't buffer the rows we end up framing every row which is
additional data and still doing 2 out of the 4 transfers we do now on data of the same size
(since the frames wouldn't be any bigger).  BTW, 2 or 4 xfes in this situation doesn't affect
the performance; the latency on the network and CPU of compaction and indexing building dwarf
any gains to be made here.  The current approach has the added benefit that debugging is easy
because it's clear where the key and row boundaries are.

{quote}What is the benefit of using an independent protocol for "Loader" streams?{quote}
If you're comparing to the streams we use for repair and similar, they require table names
and byte ranges be known up front.  While a proxy could just generate a random name, it doesn't
know the sizes because it doesn't have a SSTable on disk (or buffered in memory).  There is
also no way for a node to request a retry from a proxy if the stream fails because the proxy
won't have the data and in general is probably firewalled off C*-to-Proxy connections.  And
even if we did, we'd still have a bunch of small sessions because the proxy doesn't know when
a client is going to stop sending data to it.  In the most general sense it could be a constant
thing; a client may just continually pump an RSS feed or stock ticks or something into it.
 tl;dr simplicity and code reduction.

{quote}Again, awesome.{quote}

> Make bulk loading into Cassandra less crappy, more pluggable
> ------------------------------------------------------------
>                 Key: CASSANDRA-1278
>                 URL:
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Tools
>            Reporter: Jeremy Hanna
>            Assignee: Matthew F. Dennis
>             Fix For: 0.8.1
>         Attachments: 1278-cassandra-0.7-v2.txt, 1278-cassandra-0.7.1.txt, 1278-cassandra-0.7.txt
>   Original Estimate: 40h
>          Time Spent: 40h 40m
>  Remaining Estimate: 0h
> Currently bulk loading into Cassandra is a black art.  People are either directed to
just do it responsibly with thrift or a higher level client, or they have to explore the contrib/bmt
example -  That contrib module requires delving
into the code to find out how it works and then applying it to the given problem.  Using either
method, the user also needs to keep in mind that overloading the cluster is possible - which
will hopefully be addressed in CASSANDRA-685
> This improvement would be to create a contrib module or set of documents dealing with
bulk loading.  Perhaps it could include code in the Core to make it more pluggable for external
clients of different types.
> It is just that this is something that many that are new to Cassandra need to do - bulk
load their data into Cassandra.

This message is automatically generated by JIRA.
For more information on JIRA, see:

View raw message