hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Brock Noland (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-7195) Improve Metastore performance
Date Sun, 08 Jun 2014 21:21:01 GMT

     [ https://issues.apache.org/jira/browse/HIVE-7195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Brock Noland updated HIVE-7195:
-------------------------------

    Description: 
Even with direct SQL, which significantly improves MS performance, some operations take a
considerable amount of time, when there are many partitions on table. Specifically I believe
the issue:

* When a client gets all partitions we do not send them an iterator, we create a collection
of all data and then pass the object over the network in total

* Operations which require looking up data on the NN can still be slow since there is no cache
of information and it's done in a serial fashion

> Improve Metastore performance
> -----------------------------
>
>                 Key: HIVE-7195
>                 URL: https://issues.apache.org/jira/browse/HIVE-7195
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Brock Noland
>
> Even with direct SQL, which significantly improves MS performance, some operations take
a considerable amount of time, when there are many partitions on table. Specifically I believe
the issue:
> * When a client gets all partitions we do not send them an iterator, we create a collection
of all data and then pass the object over the network in total
> * Operations which require looking up data on the NN can still be slow since there is
no cache of information and it's done in a serial fashion



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message