hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Owen O'Malley (JIRA)" <>
Subject [jira] [Updated] (HIVE-6060) Define API for RecordUpdater and UpdateReader
Date Tue, 25 Feb 2014 01:09:27 GMT


Owen O'Malley updated HIVE-6060:

    Attachment: acid-io.patch

This is still a work in progress, but it shows the path:
* Adds AcidInputFormat and AcidOutputFormat interfaces for input/output formats that can support
the acid requirements.
* Extends OrcInputFormat and OrcOutputFormat to implement the interfaces.
* Adds AcidUtils that provides general routines to analyze the partition directory and figure
out which base and deltas to use.
* Doesn't change the behavior of the insert commands, which will still write the traditional
hive file layout. This will change later when we add command support.
* The input format in getsplits will detect whether there are new or old style layouts and
read them appropriately.
* Java clients can write the new layout by using the RecordUpdater interface.
* There are raw interfaces for the compactor to use.

> Define API for RecordUpdater and UpdateReader
> ---------------------------------------------
>                 Key: HIVE-6060
>                 URL:
>             Project: Hive
>          Issue Type: Sub-task
>            Reporter: Owen O'Malley
>            Assignee: Owen O'Malley
>         Attachments: acid-io.patch, h-5317.patch, h-5317.patch, h-5317.patch, h-6060.patch
> We need to define some new APIs for how Hive interacts with the file formats since it
needs to be much richer than the current RecordReader and RecordWriter.

This message was sent by Atlassian JIRA

View raw message