bigtop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jay vyas <>
Subject What will the next generation of bigtop look like?
Date Sat, 06 Dec 2014 23:23:30 GMT
hi bigtop !

I thought id start a thread a few vaguely related thoughts i have around
next couple iterations of bigtop.

1) Hive:  How will bigtop to evolve to support it, now that it is much
more  than a mapreduce query wrapper?

2) I wonder wether we should confirm cassandra interoperability of spark in
bigtop distros,

3) Also, as per , ---
What about presto ?  Who is interested in supporting it - packaging it
-testing it etc..?  (con) I don't know if its really ready to be in bigtop,
but (pro) i think if there is someone really dedicated to testing its
interop w/ the bigtop stack, that could be great news for us.

*** Now three concrete questions lead to a more interesting question ***

4) in general, i think bigtop can move in one of 3 directions.

  EXPAND ? : Expanding to include new components, with just basic interop,
and let folks evolve their own stacks on top of bigtop on their own.

  CONTRACT+FOCUS ?  Contracting to focus on a lean set of core components,
with super high quality.

  STAY THE COURSE ? Staying the same ~ a packaging platform for just
hadoop's direct ecosystem.

I am intrigued by the idea of A and B both have clear benefits and
costs...  would like to see the opinions of folks --- do we  lean in one
direction or another? What is the criteria for adding a new feature,
package, stack to bigtop?

... Or maybe im just overthinking it and should be spending this time
testing spark for 0.9 release....

Either way, looking forward to some feedback on these thoughts from the
bigtop community !

jay vyas

View raw message