crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Christian Tzolov (JIRA)" <>
Subject [jira] [Updated] (CRUNCH-491) Add an Xml File Source
Date Mon, 26 Jan 2015 01:03:48 GMT


Christian Tzolov updated CRUNCH-491:
    Attachment: CRUNCH-491.patch

Here is the first cut. 

Note that I've copied the HadoopCompat utility class form the Elephant-Bird project. This
utility provides a helper methods to deal with Hadoop 1.x and 2.x incompatibilities. 
It might be better to add the elephant-bird-hadoop-compat:4.6 pom dependency instead? 

> Add an Xml File Source
> ----------------------
>                 Key: CRUNCH-491
>                 URL:
>             Project: Crunch
>          Issue Type: New Feature
>          Components: Core
>    Affects Versions: 0.11.0
>            Reporter: Christian Tzolov
>            Assignee: Christian Tzolov
>            Priority: Minor
>              Labels: inputformat, source, xml
>         Attachments: CRUNCH-491.patch
> Large XML documents that are composed of a repetitive XML elements can be broken into
chunks delimited by the start and end tags of those elements.
> The XmlSource should process XML files and extract out the XML between the pre-configured
start / end tags.

This message was sent by Atlassian JIRA

View raw message