crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Wills <>
Subject Crunch release plan: 0.8.0 and HBase 0.96
Date Wed, 30 Oct 2013 17:02:03 GMT
Hey all,

We're resolved 41 issues since 0.7.0, which makes me think that it's about
time for a formal release of 0.8.0. In parallel, I've been working on
getting Crunch to work with 0.96-hadoop2 with some success. That said, the
code changes that we'll need to make to get Crunch working against the 0.96
APIs are different enough from the 0.94 APIs that I feel like maintaining
some sort of compatibility layer in our code will be pretty ugly.

I'm thinking along these lines:

1) Release 0.8.0 in the next couple of days against our current set of
dependencies (Hadoop and HBase.)
2) Upgrade the Hadoop 2 dependency to Hadoop 2.2.0, which will also require
us to upgrade to protocol buffers 2.5.0 in the build-- I've already done
this and verified that everything works.
3) Switch the HBase code to the 0.96 APIs, without trying to maintain
backwards compatibility with 0.94, and get everything working.
4) Do the 0.9.0 release against with Hadoop 2 and HBase 0.96 as the default.

I imagine that there will still be bugfixes against 0.8.0 (both core and
HBase) that will mean that we'll need to do 0.8.1, 0.8.2, etc. releases to
support, and I'm happy to keep those up at a regular cadence.


Director of Data Science
Cloudera <>
Twitter: @josh_wills <>

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message