Mailing-List: contact issues-help@hive.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hive.apache.org
Date: Tue, 24 Feb 2015 22:32:04 +0000 (UTC)
From: "Xiaobing Zhou (JIRA)" <jira@apache.org>
To: issues@hive.apache.org
Message-ID: <JIRA.12776501.1424455519000.5351.1424817124918@Atlassian.JIRA>
In-Reply-To: <JIRA.12776501.1424455519000@Atlassian.JIRA>
References: <JIRA.12776501.1424455519000@Atlassian.JIRA>
 <JIRA.12776501.1424455519554@arcas>
Subject: [jira] [Commented] (HIVE-9741) Refactor MetaStoreDirectSql
 constructor by removing DB queries out of critical section
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable


    [ https://issues.apache.org/jira/browse/HIVE-9741?page=3Dcom.atlassian.=
jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D14335=
584#comment-14335584 ]=20

Xiaobing Zhou commented on HIVE-9741:
-------------------------------------

Thanks [~sershe] for review. If setDbANSIMode() call is moved to somewhere =
like patch V4. DB query within critical section still can not be avoided as=
 the stack trace shows. This is a problem, do you have any comments?

MetaStoreDirectSql.determineDbType(...)
MetaStoreDirectSql.MetaStoreDirectSql(...)
ObjectStore.initialize(...)
ObjectStore.setConf(=E2=80=A6)


DB type detections is actually manually tested agains different backend, wo=
rks well.

> Refactor MetaStoreDirectSql constructor by removing DB queries out of cri=
tical section
> -------------------------------------------------------------------------=
-------------
>
>                 Key: HIVE-9741
>                 URL: https://issues.apache.org/jira/browse/HIVE-9741
>             Project: Hive
>          Issue Type: Bug
>          Components: Metastore
>    Affects Versions: 1.0.0
>            Reporter: Xiaobing Zhou
>            Assignee: Xiaobing Zhou
>         Attachments: HIVE-9741.1.patch, HIVE-9741.2.patch, HIVE-9741.3.pa=
tch, HIVE-9741.4.patch
>
>
> MetaStoreDirectSql constructor is querying DB to determine dbType, which =
leads to too many DB queries to make megastore slow as ObjectStore.setConf =
might be called frequently. Moreover, ObjectStore.setConf begins/ends with =
lock acquire/release, if the underlying DB hangs somehow, lock is never rel=
eased and all hereafter incoming requests are blocked.
> Two points:
> 1. Using getProductName based JDBC driver to get dbType info.
> 2. Since metastore auto-creaton is disabled by default, it'd better bypas=
s ensureDbInit() and runTestQuery() in order to avoid DB queries within cri=
tical section of setConf.
> Here=E2=80=99s stack trace:
> MetaStoreDirectSql.determineDbType(...)
> MetaStoreDirectSql.MetaStoreDirectSql(...)
> ObjectStore.initialize(...)
> ObjectStore.setConf(=E2=80=A6)


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)