hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sergey Shelukhin (JIRA)" <>
Subject [jira] [Commented] (HIVE-9741) Refactor MetaStoreDirectSql constructor by removing DB queries out of critical section
Date Tue, 24 Feb 2015 01:04:01 GMT


Sergey Shelukhin commented on HIVE-9741:

1) Why is setDbANSIMode called where it's called in ObjectStore? It should already be called
in DirectSQL before each set of queries. Note that it only works for one connection internally,
so it needs to be called in the same transaction is queries, calling it in ObjectStore doesn't
make a lot of sense.

2) The detection changes need to be tested with all the databases affected (mysql, oracle,

> Refactor MetaStoreDirectSql constructor by removing DB queries out of critical section
> --------------------------------------------------------------------------------------
>                 Key: HIVE-9741
>                 URL:
>             Project: Hive
>          Issue Type: Bug
>          Components: Metastore
>    Affects Versions: 1.0.0
>            Reporter: Xiaobing Zhou
>            Assignee: Xiaobing Zhou
>         Attachments: HIVE-9741.1.patch, HIVE-9741.2.patch, HIVE-9741.3.patch
> MetaStoreDirectSql constructor is querying DB to determine dbType, which leads to too
many DB queries to make megastore slow as ObjectStore.setConf might be called frequently.
Moreover, ObjectStore.setConf begins/ends with lock acquire/release, if the underlying DB
hangs somehow, lock is never released and all hereafter incoming requests are blocked.
> Two points:
> 1. Using getProductName based JDBC driver to get dbType info.
> 2. Since metastore auto-creaton is disabled by default, it'd better bypass ensureDbInit()
and runTestQuery() in order to avoid DB queries within critical section of setConf.
> Here’s stack trace:
> MetaStoreDirectSql.determineDbType(...)
> MetaStoreDirectSql.MetaStoreDirectSql(...)
> ObjectStore.initialize(...)
> ObjectStore.setConf(…)

This message was sent by Atlassian JIRA

View raw message