hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mithun Radhakrishnan (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-17188) ObjectStore runs out of memory for large batches of addPartitions().
Date Thu, 03 Aug 2017 17:48:00 GMT

     [ https://issues.apache.org/jira/browse/HIVE-17188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Mithun Radhakrishnan updated HIVE-17188:
----------------------------------------
    Fix Version/s: 2.4.0

> ObjectStore runs out of memory for large batches of addPartitions().
> --------------------------------------------------------------------
>
>                 Key: HIVE-17188
>                 URL: https://issues.apache.org/jira/browse/HIVE-17188
>             Project: Hive
>          Issue Type: Bug
>          Components: Metastore
>    Affects Versions: 2.2.0
>            Reporter: Mithun Radhakrishnan
>            Assignee: Chris Drome
>             Fix For: 2.2.0, 3.0.0, 2.4.0
>
>         Attachments: HIVE-17188.1.patch
>
>
> For large batches (e.g. hundreds) of {{addPartitions()}}, the {{ObjectStore}} runs out
of memory. Flushing the {{PersistenceManager}} alleviates the problem.
> Note: The problem being addressed here isn't so much with the size of the hundreds of
Partition objects, but the cruft that builds with the PersistenceManager, in the JDO layer,
as confirmed through memory-profiling.
> (Raising this on behalf of [~cdrome] and [~thiruvel].)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message