Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: pass (athena.apache.org: domain of grokdata@gmail.com designates
 74.125.82.66 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        b=R9nDupBwcBsV74FQcZFe7zish2B1BrEaNS2VjjPi+bZwVtN5/ZjYyr5WkJt+Oovu72
         0KpQ1VDFtAKoFc19yM4BlgRLeBLRiKZRqn+xjOIRxNvc4LJKqMur1av4nnP+onejZOTf
         QkNv+hi9MVEKNW0D+IkHZILBaybcfIs9ToRxo=
MIME-Version: 1.0
In-Reply-To: <1301447718.14004.74.camel@erebus.lan>
References: <AANLkTikiU7fCgTobL=3EDC-rLTu0GN7hgtzdb8sYuj0+@mail.gmail.com>
	<1301447718.14004.74.camel@erebus.lan>
Date: Thu, 31 Mar 2011 01:04:26 +0200
Message-ID: <AANLkTi=xC6+KqAbM91P6JrMog1Ft5a9CWeke9epZkCsN@mail.gmail.com>
Subject: Re: Ditching Cassandra
From: Gregori Schmidt <grokdata@gmail.com>
To: user@cassandra.apache.org
Content-Type: multipart/alternative; boundary=002618877316d1207a049fbb32f9

--002618877316d1207a049fbb32f9
Content-Type: text/plain; charset=ISO-8859-1

On Wed, Mar 30, 2011 at 3:15 AM, Eric Evans <eevans@rackspace.com> wrote:

> The client space as a whole *is* a mess, despite heroic efforts on the
> part of our third-party API maintainers, but forcing them in-tree is not
> going to solve anything.  In fact, it would very likely make it worse by
> adding unnecessary overhead to contribution, and discouraging
> innovation.
>

I can understand your reluctance to do the clients "in-tree" since it will
be a lot of work and people will no doubt be upset if their client is not
chosen for a given language.  But I think this is the wrong approach for
three reasons.

First, the client libraries in different languages should look alike.
 Having libraries that look different in different languages is a very bad
idea indeed.

Second, users should not have to ask which client library version goes with
which Cassandra version.  Nor should they have to deal with different
features appearing at different times in different libraries for different
languages. And getting everyone to version everything the same way, so users
won't have to scratch their heads, isn't going to work.

Third, if you think transitioning to maintaining official client libraries
in-tree is going to be hard and a lot of work now, just wait another year
and it is going to be even harder. Just like it is a lot harder to do now
than, say, 2 years ago. The sooner this gets done, the better.  This is
technology debt.  And if it doesn't get paid, people will go elsewhere.


> The root cause goes back to your first point, the RPC interface is
> baroque, and too tightly coupled to Cassandra's internals.  The
> third-party library maintainers can only do so much to paper over that;
> The Fail shines through.
>

Well, yes and no.  Yes the RPC interface is baroque and somewhat unhelpful.
 It worries me that nobody found it baroque and unhelpful enough to "paper
over" early on.

And no, it isn't really that hard to paper over, but the problem is that
nobody wants to write yet another Cassandra client library to provide a way
to talk to Cassandra which is both reasonably terse *and* at the same time
flexible enough to let you do more complex stuff.

One of the first urges I felt when starting to play with Cassandra was to
write my own client library.  That urge did not subside as I tried some of
the other (Java) client libraries.  (I'll be nice and just postulate that
they are roughly equally ugly so everyone is equally offended :-)).  The
problem is that writing a client library that I'd actually want to use would
not really be that helpful for the Cassandra project or for me.  It would be
yet another project competing for attention and consuming precious developer
resources.  That is: if anyone would have bothered to use it.  Either way
I'd probably get stuck maintaining my own library, or at best, have to
dedicate a considerable amount of time to coordinating a development effort
until someone actually interested in being a client lib maintainer could
take over.

Sure, having to maintain the client libraries in-tree is going to slow down
the release cycle somewhat, but which would you rather have:  more frequent
releases or a better product that appeals to developers?

And sure, there are a lot of *really* nice people that will probably feel
that they have wasted effort on building a client library if Cassandra were
to offer official client libraries,  but at this point people are beginning
to depend on Cassandra and paying attention to users (that is: programmers)
is going to be very important.  One day the Hadoop/HBase people are going to
get their shit together and *then what*.

I would much rather the client library maintainers for various libraries in
various languages teamed up with someone who is good at designing APIs that
need to work across languages, spent some time making a rough outline, did
some hacking to see what it would be like in different languages, adjusted
the common design guideline and got to work on maintaining in-tree client
libraries rather than everyone and their grandmother maintaining their own
client library.  Egos might get bruised, people may trot off in a huff, but
in the long run I think this is for the better that we do not encourage
effort to be duplicated and talent to be spread thin across different
projects.

I feel like a bit of a dick for saying things this bluntly, but I think that
someone has to be the unpopular jerk who says these things out loud.  You
have all done *great* work on Cassandra, I am grateful for it, and I am in
no way belittling your effort.  But you need an outside voice to tell you
that there are some things that aren't as they should.

Part of the problem here is that everyone is just too nice to each other
:-).  I'll be happy to be the jerk if I think it'd help the project.


> The solution here is the same as for point #1 above, CQL.  And, the idea
> is to include in-tree "drivers", basically, the minimum amount of common
> code that all third-party libs would need to implement (think connection
> pooling, parameter substitution, etc).


A minimal client for each language with a less sucky API would be a start.

Gathering the talent that is spread across different client projects today
and get them to maintain official libraries that offer some consistency
across languages would be better.

Next time you have a Cassandra summit (or similar), invite the client
library maintainers, put them in a nice room with enough fresh air, give
them a stack of pizzas and a whiteboard and let them have a couple of hours
to discuss a) if they are willing to pull together, and b) how they would go
about building better, official client libraries.  Add drugs as needed
(alcohol, red bull, carrots etc).


>  We already have drivers for
> Java, Python, and Twisted, and folks are working PHP and Perl (that I'm
> aware of).
>
> >    - It is buggy and the solution seems to be to just go to the next
> >    release.  And the next.  And the next.  Which would be okay if you
> > could upgrade all the time, but what to do once you hit production?
>
> 0.7 has been a rough ride, no doubt.  We spent too much time pushing in
> too many features, and didn't do a good job of drawing a line in the
> sand when it came time to release.  Our track record prior to 0.7 was
> Not Horrible, and trending toward Better And Better, and we've made some
> adjustment to the release process, so I'm hopeful we'll get back on
> track.
>

I'm glad to hear that.  Quality and reliability is going to make or break
Cassandra over the next 1-2 years.


> > In any case, thanks for all the effort that went into Cassandra.  I
> > will check back from time to time and perhaps in a year or so it'll be
> > time to re-evaluate Cassandra.
>
> In a year we'll have achieved Total World Domination. :)
>

Well, I sincerely hope so even though I have (for now) jumped ship :-)

But as I mentioned earlier, if the Hadoop guys get their shit together it
may be them that dominate the world.  The flock is restless and unfaithful.

~G

--002618877316d1207a049fbb32f9
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

On Wed, Mar 30, 2011 at 3:15 AM, Eric Evans <span dir=3D"ltr">&lt;<a href=
=3D"mailto:eevans@rackspace.com">eevans@rackspace.com</a>&gt;</span> wrote:=
<br><div class=3D"gmail_quote"><blockquote class=3D"gmail_quote" style=3D"m=
argin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
The client space as a whole *is* a mess, despite heroic efforts on the<br>
part of our third-party API maintainers, but forcing them in-tree is not<br=
>
going to solve anything. =A0In fact, it would very likely make it worse by<=
br>
adding unnecessary overhead to contribution, and discouraging<br>
innovation.<br></blockquote><div><br></div><div>I can understand your reluc=
tance to do the clients &quot;in-tree&quot; since it will be a lot of work =
and people will no doubt be upset if their client is not chosen for a given=
 language. =A0But I think this is the wrong approach for three reasons.</di=
v>
<div><br></div><div>First, the client libraries in different languages shou=
ld look alike. =A0Having libraries that look different in different languag=
es is a very bad idea indeed.</div><div><br></div><div>Second, users should=
 not have to ask which client library version goes with which Cassandra ver=
sion. =A0Nor should they have to deal with different features appearing at =
different times in different libraries for different languages. And getting=
 everyone to version everything the same way, so users won&#39;t have to sc=
ratch their heads, isn&#39;t going to work.</div>
<div>=A0</div><div>Third, if you think transitioning to maintaining officia=
l client libraries in-tree is going to be hard and a lot of work now, just =
wait another year and it is going to be even harder. Just like it is a lot =
harder to do now than, say, 2 years ago. The sooner this gets done, the bet=
ter. =A0This is technology debt. =A0And if it doesn&#39;t get paid, people =
will go elsewhere.</div>
<div>=A0</div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;=
border-left:1px #ccc solid;padding-left:1ex;">
The root cause goes back to your first point, the RPC interface is<br>
baroque, and too tightly coupled to Cassandra&#39;s internals. =A0The<br>
third-party library maintainers can only do so much to paper over that;<br>
The Fail shines through.<br></blockquote><div><br></div><div>Well, yes and =
no. =A0Yes the RPC interface is baroque and somewhat unhelpful. =A0It worri=
es me that nobody found it baroque and unhelpful enough to &quot;paper over=
&quot; early on. =A0</div>
<div><br></div><div>And no, it isn&#39;t really that hard to paper over, bu=
t the problem is that nobody wants to write yet another Cassandra client li=
brary to provide a way to talk to Cassandra which is both reasonably terse =
*and* at the same time flexible enough to let you do more complex stuff.</d=
iv>
<div><br></div><div>One of the first urges I felt when starting to play wit=
h Cassandra was to write my own client library. =A0That urge did not subsid=
e as I tried some of the other (Java) client libraries. =A0(I&#39;ll be nic=
e and just postulate that they are roughly equally ugly so everyone is equa=
lly offended :-)). =A0The problem is that writing a client library that I&#=
39;d actually want to use would not really be that helpful for the Cassandr=
a project or for me. =A0It would be yet another project competing for atten=
tion and consuming precious developer resources. =A0That is: if anyone woul=
d have bothered to use it. =A0Either way I&#39;d probably get stuck maintai=
ning my own library, or at best, have to dedicate a considerable amount of =
time to coordinating a development effort until someone actually interested=
 in being a client lib maintainer could take over.</div>
<div><br></div><div>Sure, having to maintain the client libraries in-tree i=
s going to slow down the release cycle somewhat, but which would you rather=
 have: =A0more frequent releases or a better product that appeals to develo=
pers?</div>
<div><br></div><div>And sure, there are a lot of *really* nice people that =
will probably feel that they have wasted effort on building a client librar=
y if Cassandra were to offer official client libraries, =A0but at this poin=
t people are beginning to depend on Cassandra and paying attention to users=
 (that is: programmers) is going to be very important. =A0One day the Hadoo=
p/HBase people are going to get their shit together and *then what*.</div>
<div><br></div><div>I would much rather the client library maintainers for =
various libraries in various languages teamed up with someone who is good a=
t designing APIs that need to work across languages, spent some time making=
 a rough outline, did some hacking to see what it would be like in differen=
t languages, adjusted the common design guideline and got to work on mainta=
ining in-tree client libraries rather than everyone and their grandmother m=
aintaining their own client library. =A0Egos might get bruised, people may =
trot off in a huff, but in the long run I think this is for the better that=
 we do not encourage effort to be duplicated and talent to be spread thin a=
cross different projects.</div>
<div><br></div><div>I feel like a bit of a dick for saying things this blun=
tly, but I think that someone has to be the unpopular jerk who says these t=
hings out loud. =A0You have all done *great* work on Cassandra, I am gratef=
ul for it, and I am in no way belittling your effort. =A0But you need an ou=
tside voice to tell you that there are some things that aren&#39;t as they =
should. =A0</div>
<div><br></div><div>Part of the problem here is that everyone is just too n=
ice to each other :-). =A0I&#39;ll be happy to be the jerk if I think it=
9;d help the project.</div><div>=A0</div><blockquote class=3D"gmail_quote" =
style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">

The solution here is the same as for point #1 above, CQL. =A0And, the idea<=
br>
is to include in-tree &quot;drivers&quot;, basically, the minimum amount of=
 common<br>
code that all third-party libs would need to implement (think connection<br=
>
pooling, parameter substitution, etc).</blockquote><div><br></div><div>A mi=
nimal client for each language with a less sucky API would be a start.</div=
><div><br></div><div>Gathering the talent that is spread across different c=
lient projects today and get them to maintain official libraries that offer=
 some consistency across languages would be better.</div>
<div><br></div><div>Next time you have a Cassandra summit (or similar), inv=
ite the client library maintainers, put them in a nice room with enough fre=
sh air, give them a stack of pizzas and a whiteboard and let them have a co=
uple of hours to discuss a) if they are willing to pull together, and b) ho=
w they would go about building better, official client libraries. =A0Add dr=
ugs as needed (alcohol, red bull, carrots etc).</div>
<div><br></div><div>=A0</div><blockquote class=3D"gmail_quote" style=3D"mar=
gin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;"> =A0We already=
 have drivers for<br>
Java, Python, and Twisted, and folks are working PHP and Perl (that I&#39;m=
<br>
aware of).<br>
<br>
&gt; =A0 =A0- It is buggy and the solution seems to be to just go to the ne=
xt<br>
<div class=3D"im">&gt; =A0 =A0release. =A0And the next. =A0And the next. =
=A0Which would be okay if you<br>
&gt; could upgrade all the time, but what to do once you hit production?<br=
>
<br>
</div>0.7 has been a rough ride, no doubt. =A0We spent too much time pushin=
g in<br>
too many features, and didn&#39;t do a good job of drawing a line in the<br=
>
sand when it came time to release. =A0Our track record prior to 0.7 was<br>
Not Horrible, and trending toward Better And Better, and we&#39;ve made som=
e<br>
adjustment to the release process, so I&#39;m hopeful we&#39;ll get back on=
<br>
track.<br></blockquote><div><br></div><div>I&#39;m glad to hear that. =A0Qu=
ality and reliability is going to make or break Cassandra over the next 1-2=
 years.</div><div>=A0</div><blockquote class=3D"gmail_quote" style=3D"margi=
n:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div class=3D"im">&gt; In any case, thanks for all the effort that went int=
o Cassandra. =A0I<br>
&gt; will check back from time to time and perhaps in a year or so it&#39;l=
l be<br>
&gt; time to re-evaluate Cassandra.<br>
<br>
</div>In a year we&#39;ll have achieved Total World Domination. :)<br></blo=
ckquote><div><br></div><div>Well, I sincerely hope so even though I have (f=
or now) jumped ship :-)</div><div><br></div><div>But as I mentioned earlier=
, if the Hadoop guys get their shit together it may be them that dominate t=
he world. =A0The flock is restless and unfaithful.</div>
<div>=A0</div><div>~G</div></div>

--002618877316d1207a049fbb32f9--