crunch-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gabriel Reid <gabriel.r...@gmail.com>
Subject Re: HCatalog Source for Crunch
Date Mon, 10 Feb 2014 15:03:14 GMT
I'm not currently in a position to make use of the HCat support, but
it definitely sounds like a cool thing to have in Crunch.

I also agree that it should be in a separate Maven module.

- Gabriel

On Mon, Feb 10, 2014 at 8:17 AM, Josh Wills <jwills@cloudera.com> wrote:
> Hey all,
>
> I wanted to solicit feedback on
> https://issues.apache.org/jira/browse/CRUNCH-340, which adds HCatalog
> Source and Target types to Crunch. I'm of two minds about adding in support
> for HCatalog to the core project; in general, I like for their to be more
> ways to read and write data from Crunch, esp. when we can add
> interoperability w/Hive, which is such a natural complement to Crunch. On
> the other hand, I'm concerned about the costs that Hive's not-super-nice
> dependency framework impose on Crunch clients and the project-- that is to
> say, I'm worried that bringing in HCat dependencies makes it harder for us
> to update existing dependencies and add in new dependencies that will
> conflict with the Hive/HCat dependencies. I'd like to hear from at least a
> few folks that the HCat support would be useful for them before we promote
> it from a useful extension to a feature of the core project.
>
> Thanks!
> Josh
>
> --
> Director of Data Science
> Cloudera <http://www.cloudera.com>
> Twitter: @josh_wills <http://twitter.com/josh_wills>

Mime
View raw message