crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matthias Friedrich <m...@mafr.de>
Subject Re: HCatalog Source for Crunch
Date Mon, 10 Feb 2014 08:50:19 GMT
Hi,

as someone who had to deal with dependency hell many times, I'd really
appreciate a separate Maven module for HCat.

Regards,
  Matthias

On Sunday, 2014-02-09, Josh Wills wrote:
> Hey all,
> 
> I wanted to solicit feedback on
> https://issues.apache.org/jira/browse/CRUNCH-340, which adds HCatalog
> Source and Target types to Crunch. I'm of two minds about adding in support
> for HCatalog to the core project; in general, I like for their to be more
> ways to read and write data from Crunch, esp. when we can add
> interoperability w/Hive, which is such a natural complement to Crunch. On
> the other hand, I'm concerned about the costs that Hive's not-super-nice
> dependency framework impose on Crunch clients and the project-- that is to
> say, I'm worried that bringing in HCat dependencies makes it harder for us
> to update existing dependencies and add in new dependencies that will
> conflict with the Hive/HCat dependencies. I'd like to hear from at least a
> few folks that the HCat support would be useful for them before we promote
> it from a useful extension to a feature of the core project.
> 
> Thanks!
> Josh
> 
> -- 
> Director of Data Science
> Cloudera <http://www.cloudera.com>
> Twitter: @josh_wills <http://twitter.com/josh_wills>

Mime
View raw message