hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Prasanth J (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-7250) Adaptive compression buffer size for wide tables in ORC
Date Thu, 19 Jun 2014 18:50:26 GMT

     [ https://issues.apache.org/jira/browse/HIVE-7250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Prasanth J updated HIVE-7250:
-----------------------------

    Attachment: HIVE-7250.5.patch

Added missing apache license header in the newly added unit tests. Also made the unit tests
not dependant on -Xmx

> Adaptive compression buffer size for wide tables in ORC
> -------------------------------------------------------
>
>                 Key: HIVE-7250
>                 URL: https://issues.apache.org/jira/browse/HIVE-7250
>             Project: Hive
>          Issue Type: Improvement
>          Components: File Formats
>    Affects Versions: 0.14.0
>            Reporter: Prasanth J
>            Assignee: Prasanth J
>              Labels: orcfile
>         Attachments: HIVE-7250.1.patch, HIVE-7250.2.patch, HIVE-7250.3.patch, HIVE-7250.4.patch,
HIVE-7250.5.patch
>
>
> If the input table is wide (in the order of 1000s), ORC compression buffer size overhead
becomes significant causing OOM issues. To overcome this issue, buffer size should be adaptively
chosen based on the available memory and the number of columns.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message