hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "XIAOBING ZHOU" <xzho...@gmail.com>
Subject Re: Review Request 31249: Refactor MetaStoreDirectSql constructor by removing DB queries out of critical section
Date Tue, 24 Feb 2015 00:49:15 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31249/
-----------------------------------------------------------

(Updated Feb. 24, 2015, 12:49 a.m.)


Review request for hive.


Summary (updated)
-----------------

Refactor MetaStoreDirectSql constructor by removing DB queries out of critical section


Repository: hive-git


Description
-------

MetaStoreDirectSql constructor is querying DB to determine dbType, which leads to too many
DB queries to make megastore slow as ObjectStore.setConf might be called frequently. Moreover,
ObjectStore.setConf begins/ends with lock acquire/release, if the underlying DB hangs somehow,
lock is never released and all hereafter incoming requests are blocked.

Two points:
1. Using getProductName based JDBC driver to get dbType info.
2. Since metastore auto-creaton is disabled by default, it'd better bypass ensureDbInit()
and runTestQuery() in order to avoid DB queries within critical section of setConf.

Here’s stack trace:
MetaStoreDirectSql.determineDbType(...)
MetaStoreDirectSql.MetaStoreDirectSql(...)
ObjectStore.initialize(...)
ObjectStore.setConf(…)


Diffs
-----

  metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 574141c 
  metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java fcaffc7 

Diff: https://reviews.apache.org/r/31249/diff/


Testing
-------


Thanks,

XIAOBING ZHOU


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message