hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yuxing Yao (JIRA)" <>
Subject [jira] [Created] (HIVE-15290) Stripe size smaller than specified.
Date Mon, 28 Nov 2016 08:20:58 GMT
Yuxing Yao created HIVE-15290:

             Summary: Stripe size smaller than specified.
                 Key: HIVE-15290
             Project: Hive
          Issue Type: Bug
          Components: ORC
    Affects Versions: 2.0.1, 2.1.0, 2.0.0, 1.2.1, 1.2.0
            Reporter: Yuxing Yao

In Hive-1.2.0, the real stripe size of output orc file will be very small if most of table
data are empty, result in too many Column Statistics objects consumes most of the memory.
I found it become better in Hive-2.0.1, but the stripe size still much smaller than expected.
I saw there's a Jira item: moved the compressed
= null out of if block, this changes helps a lot, but for completely fix this, another change
is needed in `OutStream.getBufferSize()`
I've created the PR:
Please take a look.

This message was sent by Atlassian JIRA

View raw message