hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Phabricator (Updated) (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-2907) Hive error when dropping a table with large number of partitions
Date Mon, 26 Mar 2012 19:00:31 GMT

     [ https://issues.apache.org/jira/browse/HIVE-2907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Phabricator updated HIVE-2907:
------------------------------

    Attachment: HIVE-2907.D2505.1.patch

mousom requested code review of "HIVE-2907 [jira] Hive error when dropping a table with large
number of partitions".
Reviewers: JIRA

  https://issues.apache.org/jira/browse/HIVE-2907

  Hive error when dropping a table with large number of partitions.

  Running into an "Out Of Memory" error when trying to drop a table with 128K partitions.

  The methods dropTable in metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java
  and dropTable in ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java encounter out of
memory errors
  when dropping tables with lots of partitions because they try to load the metadata for every
partition into memory.

TEST PLAN
  EMPTY

REVISION DETAIL
  https://reviews.facebook.net/D2505

AFFECTED FILES
  metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java
  metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java
  metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java
  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
  ql/src/test/results/clientnegative/drop_table_failure3.q.out
  ql/src/test/queries/clientnegative/drop_table_failure3.q
  ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java

MANAGE HERALD DIFFERENTIAL RULES
  https://reviews.facebook.net/herald/view/differential/

WHY DID I GET THIS EMAIL?
  https://reviews.facebook.net/herald/transcript/5595/

Tip: use the X-Herald-Rules header to filter Herald messages in your client.

                
> Hive error when dropping a table with large number of partitions
> ----------------------------------------------------------------
>
>                 Key: HIVE-2907
>                 URL: https://issues.apache.org/jira/browse/HIVE-2907
>             Project: Hive
>          Issue Type: Bug
>          Components: Metastore
>    Affects Versions: 0.9.0
>         Environment: General. Hive Metastore bug.
>            Reporter: Mousom Dhar Gupta
>            Priority: Minor
>             Fix For: 0.9.0
>
>         Attachments: HIVE-2907.1.patch.txt, HIVE-2907.D2505.1.patch
>
>   Original Estimate: 10h
>  Remaining Estimate: 10h
>
> Running into an "Out Of Memory" error when trying to drop a table with 128K partitions.
> The methods dropTable in metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java

> and dropTable in ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java encounter out
of memory errors 
> when dropping tables with lots of partitions because they try to load the metadata for
every partition into memory.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message