hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Dai (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HIVE-15936) ConcurrentModificationException in ATSHook
Date Thu, 16 Feb 2017 02:11:41 GMT
Daniel Dai created HIVE-15936:
---------------------------------

             Summary: ConcurrentModificationException in ATSHook
                 Key: HIVE-15936
                 URL: https://issues.apache.org/jira/browse/HIVE-15936
             Project: Hive
          Issue Type: Bug
            Reporter: Daniel Dai
            Assignee: Daniel Dai
         Attachments: HIVE-15936.1.patch

See ATSHook error:

{noformat}
java.util.ConcurrentModificationException
	at java.util.HashMap$HashIterator.nextNode(HashMap.java:1437) ~[?:1.8.0_112]
	at java.util.HashMap$EntryIterator.next(HashMap.java:1471) ~[?:1.8.0_112]
	at java.util.HashMap$EntryIterator.next(HashMap.java:1469) ~[?:1.8.0_112]
	at java.util.AbstractCollection.toArray(AbstractCollection.java:196) ~[?:1.8.0_112]
	at com.google.common.collect.ImmutableMap.copyOf(ImmutableMap.java:290) ~[guava-14.0.1.jar:?]
	at org.apache.hadoop.hive.ql.log.PerfLogger.getEndTimes(PerfLogger.java:219) ~[hive-common-2.1.0.2.6.0.0-457.jar:2.1.0.2.6.0.0-457]
	at org.apache.hadoop.hive.ql.hooks.ATSHook.createPostHookEvent(ATSHook.java:347) ~[hive-exec-2.1.0.2.6.0.0-457.jar:2.1.0.2.6.0.0-457]
	at org.apache.hadoop.hive.ql.hooks.ATSHook$2.run(ATSHook.java:206) [hive-exec-2.1.0.2.6.0.0-457.jar:2.1.0.2.6.0.0-457]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_112]
	at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_112]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_112]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_112]
	at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112]
{noformat}

According to [~jdere], ATSHook is currently accessing the PerfLogger on a separate thread,
which means the main query thread can potentially write to the PerfLogger at the same time.
The ATSHook should access the PerfLogger on the main query thread, before it sends the execution
to the ATS Logger thread.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message