hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hive QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-18776) MaterializationsInvalidationCache loading causes race condition in the metastore
Date Fri, 23 Feb 2018 21:20:04 GMT

    [ https://issues.apache.org/jira/browse/HIVE-18776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16374978#comment-16374978
] 

Hive QA commented on HIVE-18776:
--------------------------------

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  0s{color} | {color:blue}
Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  1s{color} |
{color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  7m 28s{color}
| {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 39s{color} |
{color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 22s{color}
| {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 48s{color} |
{color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 44s{color}
| {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 37s{color} |
{color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 37s{color} | {color:green}
the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 19s{color} | {color:red}
standalone-metastore: The patch generated 2 new + 546 unchanged - 1 fixed = 548 total (was
547) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m  0s{color}
| {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 45s{color} |
{color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red}  0m 13s{color} | {color:red}
The patch generated 49 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 12m 14s{color} | {color:black}
{color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03)
x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-9338/dev-support/hive-personality.sh
|
| git revision | master / ed487ac |
| Default Java | 1.8.0_111 |
| checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-9338/yetus/diff-checkstyle-standalone-metastore.txt
|
| asflicense | http://104.198.109.242/logs//PreCommit-HIVE-Build-9338/yetus/patch-asflicense-problems.txt
|
| modules | C: standalone-metastore U: standalone-metastore |
| Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-9338/yetus.txt |
| Powered by | Apache Yetus    http://yetus.apache.org |


This message was automatically generated.



> MaterializationsInvalidationCache loading causes race condition in the metastore
> --------------------------------------------------------------------------------
>
>                 Key: HIVE-18776
>                 URL: https://issues.apache.org/jira/browse/HIVE-18776
>             Project: Hive
>          Issue Type: Bug
>          Components: Materialized views, Metastore
>    Affects Versions: 3.0.0
>            Reporter: Alan Gates
>            Assignee: Jesus Camacho Rodriguez
>            Priority: Major
>         Attachments: HIVE-18776.patch
>
>
> I am seeing occasional failures running metastore tests where operations are failing
saying that there is no open transaction.  I have traced this to a race condition in loading
the materialized view invalidation cache.  When it is initialized (either in HiveMetaStoreClient
in embedded mode or in HiveMetaStore in remote mode) it grabs a copy of the current RawStore
instance and then loads the cache in a separate thread.  But ObjectStore keeps state regarding
JDO transactions with the underlying RDBMS.  So with the loader thread and the initial thread
both doing operations against the RawStore they sometimes mess up each others transaction
stack.  In a quick test I used HMSHandler.newRawStoreForConf() to fix this, which seemed
to work.
> A reference to the TxnHandler is also called.  I suspect this will run into a similar
issue.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message