cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stefania (JIRA)" <>
Subject [jira] [Commented] (CASSANDRA-12269) Faster write path
Date Mon, 25 Jul 2016 10:37:20 GMT


Stefania commented on CASSANDRA-12269:

Sylvain has definitely picked up the most important points, especially breaking of encapsulation,
I would really rather avoid the following:

assert row instanceof BTreeRow;
Object[] btree = ((BTreeRow)row).btree;

Here are some other points, in addition to those raised above by Sylvain:

* We can get rid of {{WrappedException}} if we define this interface:

public interface ApplyFunction<T, E extends Exception>
    void accept(T t) throws E;

and change the signatures to:

public static <V, E extends Exception> void applyForwards(Object[] btree, ApplyFunction<V,
E> function, Predicate<V> stopCondition) throws E

* Rather than creating utility classes for setting values we can use references or optionals
or, better, declare a small class that implements 
the interface and store these values as class fields. This should also help in debugging and

* Isn't 1 MB a bit too much for {{MAX_RECYCLE_BUFFER_SIZE}} in DataOutputBuffer? We have lots
of threads at the moment, WDYT?

* We got one weird failure (StaticColumnsTest) and several timeouts in the unit tests, it's
better to rebase and rerun to try and shake some of the failures.

* How do we run MutationBench? I got this error when running it from IntelliJ: {{java.lang.RuntimeException:
ERROR: Unable to find the resource: /META-INF/BenchmarkList}}

* I spent a bit of time trying to understand {{BTree.applyReverseInner()}} and eventually
I got there, mostly because I was not familiar with how the values are stored in BTree. It's
a matter of personal preference but, for me it would be clearer if the index in {{BTree.applyReverseInner()}}
was calculated with the exact mirror of {{BTree.applyForwardInner()}}, therefore {{int idx
= isLeaf ? i : i + (visited / 2) - (i % 2 == 0 ? 0 : childOffset);}} but leaving some comments
albeit condensed in one paragraph just above that line. Really up to you though.



* Unused imports in,,,,

* Two identical typos in documentation: {{till a stop condition it reached}} ->
{{till a stop condition is reached}}

* Maybe a comment for {{FastThreadExecutor}} explaining that this class is used by {{MutationBench}}
via {{-Djmh.executor.class}} would be useful.

> Faster write path
> -----------------
>                 Key: CASSANDRA-12269
>                 URL:
>             Project: Cassandra
>          Issue Type: Improvement
>            Reporter: T Jake Luciani
>            Assignee: T Jake Luciani
>             Fix For: 3.10
> The new storage engine (CASSANDRA-8099) has caused a regression in write performance.
 This ticket is to address it and bring 3.0 as close to 2.2 as possible. There are four main
reasons for this I've discovered after much toil:
> 1.  The cost of calculating the size of a serialized row is higher now since we no longer
have the cell name and value managed as ByteBuffers as we did pre-3.0.  That means we current
re-serialize the row twice, once to calculate the size and once to write the data.  This happens
during the SSTable writes and was addressed in CASSANDRA-9766.
>      Double serialization is also happening in CommitLog and the MessagingService.  We
need to apply the same techniques to these as we did to the SSTable serialization.
> 2.  Even after fixing (1) there is still an issue with there being more GC pressure and
CPU usage in 3.0 due to the fact that we encode everything from the {{Column}} to the {{Row}}
to the {{Partition}} as a {{BTree}}.  Specifically, the {{BTreeSearchIterator}} is used for
all iterator() methods.  Both these classes are useful for efficient removal and searching
of the trees but in the case of SerDe we almost always want to simply walk the entire tree
forwards or reversed and apply a function to each element.  To that end, we can use lambdas
and do this without any extra classes.
> 3.  We use a lot of thread locals and check them constantly on the read/write paths.
 For client warnings, tracing, temp buffers, etc.  We should move all thread locals to FastThreadLocals
and threads to FastThreadLocalThreads.
> 4.  We changed the memtable flusher defaults in 3.2 that caused a regression see: CASSANDRA-12228

This message was sent by Atlassian JIRA

View raw message