hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Phabricator (JIRA)" <>
Subject [jira] [Commented] (HIVE-4221) Stripe-level merge for ORC files
Date Thu, 25 Apr 2013 00:41:14 GMT


Phabricator commented on HIVE-4221:

sxyuan has commented on the revision "HIVE-4221 [jira] Stripe-level merge for ORC files

  common/src/java/org/apache/hadoop/hive/conf/ Will do.
  ql/src/java/org/apache/hadoop/hive/ql/io/merge/ Will do.
  ql/src/test/queries/clientpositive/orcfile_merge1.q:1 I think setting the max split size
to 100 should generate multiple splits despite the small input. I can check that a merge is
actually happening and add comments for the tests.
  ql/src/test/queries/clientpositive/orcfile_merge2.q:1 As above.


To: kevinwilfong, omalley, sxyuan

> Stripe-level merge for ORC files
> --------------------------------
>                 Key: HIVE-4221
>                 URL:
>             Project: Hive
>          Issue Type: Improvement
>          Components: Query Processor
>            Reporter: Samuel Yuan
>            Assignee: Samuel Yuan
>         Attachments: HIVE-4221.HIVE-4221.HIVE-4221.HIVE-4221.D9759.1.patch
> As with RC files, we would like to be able to merge ORC files efficiently by reading/writing
stripes without decompressing/recompressing them. This will be similar to the RC file merge,
except that footers will have to be updated with the stripe positions in the new file.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

View raw message