cassandra-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jason Brown <jasedbr...@gmail.com>
Subject Re: Cassandra on RocksDB experiment result
Date Thu, 20 Apr 2017 23:36:11 GMT
I'm +1 on the idea of a pluggable storage engine. There's clearly a
bandwidth problem for developing/reviewing/maintain multiple storage
engines, but I think having the interface is a good thing and can enhance
testability.

At a minimum I think it's worthwhile to explore the storage engine
interface, although it may turn out that it's infeasible/impractical given
the current system. And that's OK.

Thanks,

-Jason


On Thu, Apr 20, 2017 at 4:25 PM, Jeff Jirsa <jjirsa@gmail.com> wrote:

> Let's try to make this actionable. Long time
> contributors/committers/members of the PMC (especially you guys who have
> been working on internals for 4-8 years):
>
> Setting aside details of the implementation, does anyone feel that
> pluggable storage in itself is inherently a bad idea (so much so that you'd
> -1 it if someone else did the work)?
>
> If we can establish loose consensus on it being something generally
> acceptable (assuming someone can come up with an interface/abstraction upon
> which everyone can agree), then it seems like the next step is working on
> defining the proper interface.
>
> - Jeff
>
>
> On Wed, Apr 19, 2017 at 9:21 AM, Dikang Gu <dikang85@gmail.com> wrote:
>
> > Hi Cassandra developers,
> >
> > This is Dikang from Instagram, I'd like to share you some experiment
> > results we did recently, to use RocksDB as Cassandra's storage engine. In
> > the experiment, I built a prototype to integrate Cassandra 3.0.12 and
> > RocksDB on single column (key-value) use case, shadowed one of our
> > production use case, and saw about 4-6X P99 read latency drop during peak
> > time, compared to 3.0.12. Also, the P99 latency became more predictable
> as
> > well.
> >
> > Here is detailed note with more metrics:
> >
> > https://docs.google.com/document/d/1Ztqcu8Jzh4USKoWBgDJQw82DBurQm
> > sV-PmfiJYvu_Dc/edit?usp=sharing
> >
> > Please take a look and let me know your thoughts. I think the biggest
> > latency win comes from we get rid of most Java garbages created by
> current
> > read/write path and compactions, which reduces the JVM overhead and makes
> > the latency to be more predictable.
> >
> > We are very excited about the potential performance gain. As the next
> step,
> > I propose to make the Cassandra storage engine to be pluggable (like
> Mysql
> > and MongoDB), and we are very interested in providing RocksDB as one
> > storage option with more predictable performance, together with
> community.
> >
> > Thanks.
> >
> > --
> > Dikang
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message