hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bennie Schut (JIRA)" <>
Subject [jira] Commented: (HIVE-1539) Concurrent metastore threading problem
Date Mon, 16 Aug 2010 07:02:16 GMT


Bennie Schut commented on HIVE-1539:

This is starting to look more and more like a similar problem we have on the jdbc driver (HIVE-1428).
the HiveMetatStoreClient is doing calls on the thrift client but the thrift client isn't threadsafe
(probably by design). So when the HiveMetaStoreClient is used it should be synchronized or
a new HiveMetaStoreClient object should be created for each thread.

>From the "Hive" class:

  private IMetaStoreClient getMSC() throws MetaException {
    if (metaStoreClient == null) {
      metaStoreClient = createMetaStoreClient();
    return metaStoreClient;

I could make the sync factory we use on HIVE-1428 a common thing and reuse it here. Any objections?

> Concurrent metastore threading problem 
> ---------------------------------------
>                 Key: HIVE-1539
>                 URL:
>             Project: Hadoop Hive
>          Issue Type: Bug
>          Components: Metastore
>    Affects Versions: 0.7.0
>            Reporter: Bennie Schut
>            Assignee: Bennie Schut
>         Attachments: thread_dump_hanging.txt
> When running hive as a service and running a high number of queries concurrently I end
up with multiple threads running at 100% cpu without any progress.
> Looking at these threads I notice this thread(484e):
> at org.apache.hadoop.hive.metastore.ObjectStore.getMTable(
> But on a different thread(63a2):
> at org.apache.hadoop.hive.metastore.model.MStorageDescriptor.jdoReplaceField(

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message