crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Wills (JIRA)" <>
Subject [jira] [Updated] (CRUNCH-340) HCatSource
Date Sun, 08 Feb 2015 00:05:35 GMT


Josh Wills updated CRUNCH-340:
    Attachment: CRUNCH-340.patch

Did my own revision today as part of another project I'm working on-- just a simple Source
right now, I haven't implemented ReadableSource yet. In order to make this work w/both hadoop1
and hadoop2 I had to lift a pretty healthy fraction of the o.a.hive.hcatalog.mapreduce code,
since Hive uses its own shims that aren't compatible w/Crunch profiles.

> HCatSource
> ----------
>                 Key: CRUNCH-340
>                 URL:
>             Project: Crunch
>          Issue Type: New Feature
>            Reporter: Chao Shi
>         Attachments: CRUNCH-340.patch, crunch-340-v2.patch, crunch-340-v3.patch, crunch-340-v4.patch,
> This patch adds HCatSource, which enables crunch pipeline to read from Hive tables. This
is the very first version, leaving a few TODOs in code.
> It adds new dependency from crunch-core to hcatalog (as well as several hive components).
I guess maybe we should create a new subproject (e.g. crunch-hcatalog) rather than add it
into crunch-core.

This message was sent by Atlassian JIRA

View raw message