incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gregori Schmidt <>
Subject Re: Ditching Cassandra
Date Wed, 30 Mar 2011 23:04:26 GMT
On Wed, Mar 30, 2011 at 3:15 AM, Eric Evans <> wrote:

> The client space as a whole *is* a mess, despite heroic efforts on the
> part of our third-party API maintainers, but forcing them in-tree is not
> going to solve anything.  In fact, it would very likely make it worse by
> adding unnecessary overhead to contribution, and discouraging
> innovation.

I can understand your reluctance to do the clients "in-tree" since it will
be a lot of work and people will no doubt be upset if their client is not
chosen for a given language.  But I think this is the wrong approach for
three reasons.

First, the client libraries in different languages should look alike.
 Having libraries that look different in different languages is a very bad
idea indeed.

Second, users should not have to ask which client library version goes with
which Cassandra version.  Nor should they have to deal with different
features appearing at different times in different libraries for different
languages. And getting everyone to version everything the same way, so users
won't have to scratch their heads, isn't going to work.

Third, if you think transitioning to maintaining official client libraries
in-tree is going to be hard and a lot of work now, just wait another year
and it is going to be even harder. Just like it is a lot harder to do now
than, say, 2 years ago. The sooner this gets done, the better.  This is
technology debt.  And if it doesn't get paid, people will go elsewhere.

> The root cause goes back to your first point, the RPC interface is
> baroque, and too tightly coupled to Cassandra's internals.  The
> third-party library maintainers can only do so much to paper over that;
> The Fail shines through.

Well, yes and no.  Yes the RPC interface is baroque and somewhat unhelpful.
 It worries me that nobody found it baroque and unhelpful enough to "paper
over" early on.

And no, it isn't really that hard to paper over, but the problem is that
nobody wants to write yet another Cassandra client library to provide a way
to talk to Cassandra which is both reasonably terse *and* at the same time
flexible enough to let you do more complex stuff.

One of the first urges I felt when starting to play with Cassandra was to
write my own client library.  That urge did not subside as I tried some of
the other (Java) client libraries.  (I'll be nice and just postulate that
they are roughly equally ugly so everyone is equally offended :-)).  The
problem is that writing a client library that I'd actually want to use would
not really be that helpful for the Cassandra project or for me.  It would be
yet another project competing for attention and consuming precious developer
resources.  That is: if anyone would have bothered to use it.  Either way
I'd probably get stuck maintaining my own library, or at best, have to
dedicate a considerable amount of time to coordinating a development effort
until someone actually interested in being a client lib maintainer could
take over.

Sure, having to maintain the client libraries in-tree is going to slow down
the release cycle somewhat, but which would you rather have:  more frequent
releases or a better product that appeals to developers?

And sure, there are a lot of *really* nice people that will probably feel
that they have wasted effort on building a client library if Cassandra were
to offer official client libraries,  but at this point people are beginning
to depend on Cassandra and paying attention to users (that is: programmers)
is going to be very important.  One day the Hadoop/HBase people are going to
get their shit together and *then what*.

I would much rather the client library maintainers for various libraries in
various languages teamed up with someone who is good at designing APIs that
need to work across languages, spent some time making a rough outline, did
some hacking to see what it would be like in different languages, adjusted
the common design guideline and got to work on maintaining in-tree client
libraries rather than everyone and their grandmother maintaining their own
client library.  Egos might get bruised, people may trot off in a huff, but
in the long run I think this is for the better that we do not encourage
effort to be duplicated and talent to be spread thin across different

I feel like a bit of a dick for saying things this bluntly, but I think that
someone has to be the unpopular jerk who says these things out loud.  You
have all done *great* work on Cassandra, I am grateful for it, and I am in
no way belittling your effort.  But you need an outside voice to tell you
that there are some things that aren't as they should.

Part of the problem here is that everyone is just too nice to each other
:-).  I'll be happy to be the jerk if I think it'd help the project.

> The solution here is the same as for point #1 above, CQL.  And, the idea
> is to include in-tree "drivers", basically, the minimum amount of common
> code that all third-party libs would need to implement (think connection
> pooling, parameter substitution, etc).

A minimal client for each language with a less sucky API would be a start.

Gathering the talent that is spread across different client projects today
and get them to maintain official libraries that offer some consistency
across languages would be better.

Next time you have a Cassandra summit (or similar), invite the client
library maintainers, put them in a nice room with enough fresh air, give
them a stack of pizzas and a whiteboard and let them have a couple of hours
to discuss a) if they are willing to pull together, and b) how they would go
about building better, official client libraries.  Add drugs as needed
(alcohol, red bull, carrots etc).

>  We already have drivers for
> Java, Python, and Twisted, and folks are working PHP and Perl (that I'm
> aware of).
> >    - It is buggy and the solution seems to be to just go to the next
> >    release.  And the next.  And the next.  Which would be okay if you
> > could upgrade all the time, but what to do once you hit production?
> 0.7 has been a rough ride, no doubt.  We spent too much time pushing in
> too many features, and didn't do a good job of drawing a line in the
> sand when it came time to release.  Our track record prior to 0.7 was
> Not Horrible, and trending toward Better And Better, and we've made some
> adjustment to the release process, so I'm hopeful we'll get back on
> track.

I'm glad to hear that.  Quality and reliability is going to make or break
Cassandra over the next 1-2 years.

> > In any case, thanks for all the effort that went into Cassandra.  I
> > will check back from time to time and perhaps in a year or so it'll be
> > time to re-evaluate Cassandra.
> In a year we'll have achieved Total World Domination. :)

Well, I sincerely hope so even though I have (for now) jumped ship :-)

But as I mentioned earlier, if the Hadoop guys get their shit together it
may be them that dominate the world.  The flock is restless and unfaithful.


View raw message