Mailing-List: contact dev-help@hive.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hive.apache.org
Date: Thu, 29 Aug 2013 21:41:51 +0000 (UTC)
From: "Phabricator (JIRA)" <jira@apache.org>
To: hive-dev@hadoop.apache.org
Message-ID: <JIRA.12662706.1375997043829.56599.1377812511691@arcas>
In-Reply-To: <JIRA.12662706.1375997043829@arcas>
References: <JIRA.12662706.1375997043829@arcas>
Subject: [jira] [Commented] (HIVE-5029) direct SQL perf optimization cannot
 be tested well
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HIVE-5029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13754093#comment-13754093 ] 

Phabricator commented on HIVE-5029:
-----------------------------------

sershe has commented on the revision "HIVE-5029 [jira] direct SQL perf optimization cannot be tested well".

INLINE COMMENTS
  metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java:1684 the rollback is performed in finally. Here we only roll back to re-open it
  metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java:1783 the rollback is performed in finally. Here we only roll back to re-open it

REVISION DETAIL
  https://reviews.facebook.net/D12483

To: JIRA, ashutoshc, sershe

                
> direct SQL perf optimization cannot be tested well
> --------------------------------------------------
>
>                 Key: HIVE-5029
>                 URL: https://issues.apache.org/jira/browse/HIVE-5029
>             Project: Hive
>          Issue Type: Test
>            Reporter: Sergey Shelukhin
>            Assignee: Sergey Shelukhin
>            Priority: Critical
>         Attachments: HIVE-5029.D12483.1.patch, HIVE-5029.D12483.2.patch, HIVE-5029.patch, HIVE-5029.patch
>
>
> HIVE-4051 introduced perf optimization that involves getting partitions directly via SQL in metastore. Given that SQL queries might not work on all datastores (and will not work on non-SQL ones), JDO fallback is in place.
> Given that perf improvement is very large for short queries, it's on by default.
> However, there's a problem with tests with regard to that. If SQL code is broken, tests may fall back to JDO and pass. If JDO code is broken, SQL might allow tests to pass.
> We are going to disable SQL by default before the testing problem is resolved.
> There are several possible solultions:
> 1) Separate build for this setting. Seems like an overkill...
> 2) Enable by default; disable by default in tests, create a clone of TestCliDriver with a subset of queries that will exercise the SQL path.
> 3) Have some sort of test hook inside metastore that will run both ORM and SQL and compare.
> 3') Or make a subclass of ObjectStore that will do that. ObjectStore is already pluggable.
> 4) Write unit tests for one of the modes (JDO, as non-default?) and declare that they are sufficient; disable fallback in tests.
> 3' seems like the easiest. For now we will disable SQL by default.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira