hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sergey Shelukhin (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-13002) metastore call timing is not threadsafe
Date Mon, 22 Feb 2016 22:13:18 GMT

    [ https://issues.apache.org/jira/browse/HIVE-13002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15157817#comment-15157817
] 

Sergey Shelukhin commented on HIVE-13002:
-----------------------------------------

There are ~12 places where Hive object is stored in a field. The most prominent are BaseSemanticAnalyzer
and Task, where they are obviously used by 10000 methods/subclasses. The Task is esp. suspect
since many of these would be executed on different threads. There's also one in HS2 SessionState,
which as far as I know is not tied to the thread. There are also other less prominent places
where it can happen. We can fix this issue and file a follow-up JIRA to clean up all this
mess.

> metastore call timing is not threadsafe
> ---------------------------------------
>
>                 Key: HIVE-13002
>                 URL: https://issues.apache.org/jira/browse/HIVE-13002
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Sergey Shelukhin
>            Assignee: Sergey Shelukhin
>         Attachments: HIVE-13002.01.patch, HIVE-13002.patch
>
>
> Discovered in some q test run:
> {noformat}
>  TestCliDriver.testCliDriver_insert_values_orig_table:123->runTest:199 Unexpected
exception java.util.ConcurrentModificationException
> 	at java.util.HashMap$HashIterator.nextEntry(HashMap.java:926)
> 	at java.util.HashMap$EntryIterator.next(HashMap.java:966)
> 	at java.util.HashMap$EntryIterator.next(HashMap.java:964)
> 	at org.apache.hadoop.hive.ql.metadata.Hive.dumpAndClearMetaCallTiming(Hive.java:3412)
> 	at org.apache.hadoop.hive.ql.Driver.dumpMetaCallTimingWithoutEx(Driver.java:574)
> 	at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1722)
> 	at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1342)
> 	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1113)
> 	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1101)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message