crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matthias Friedrich <>
Subject Re: HCatalog Source for Crunch
Date Mon, 10 Feb 2014 08:50:19 GMT

as someone who had to deal with dependency hell many times, I'd really
appreciate a separate Maven module for HCat.


On Sunday, 2014-02-09, Josh Wills wrote:
> Hey all,
> I wanted to solicit feedback on
>, which adds HCatalog
> Source and Target types to Crunch. I'm of two minds about adding in support
> for HCatalog to the core project; in general, I like for their to be more
> ways to read and write data from Crunch, esp. when we can add
> interoperability w/Hive, which is such a natural complement to Crunch. On
> the other hand, I'm concerned about the costs that Hive's not-super-nice
> dependency framework impose on Crunch clients and the project-- that is to
> say, I'm worried that bringing in HCat dependencies makes it harder for us
> to update existing dependencies and add in new dependencies that will
> conflict with the Hive/HCat dependencies. I'd like to hear from at least a
> few folks that the HCat support would be useful for them before we promote
> it from a useful extension to a feature of the core project.
> Thanks!
> Josh
> -- 
> Director of Data Science
> Cloudera <>
> Twitter: @josh_wills <>

View raw message