crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Wills <>
Subject HCatalog Source for Crunch
Date Mon, 10 Feb 2014 07:17:47 GMT
Hey all,

I wanted to solicit feedback on, which adds HCatalog
Source and Target types to Crunch. I'm of two minds about adding in support
for HCatalog to the core project; in general, I like for their to be more
ways to read and write data from Crunch, esp. when we can add
interoperability w/Hive, which is such a natural complement to Crunch. On
the other hand, I'm concerned about the costs that Hive's not-super-nice
dependency framework impose on Crunch clients and the project-- that is to
say, I'm worried that bringing in HCat dependencies makes it harder for us
to update existing dependencies and add in new dependencies that will
conflict with the Hive/HCat dependencies. I'd like to hear from at least a
few folks that the HCat support would be useful for them before we promote
it from a useful extension to a feature of the core project.


Director of Data Science
Cloudera <>
Twitter: @josh_wills <>

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message