Return-Path: X-Original-To: apmail-hive-dev-archive@www.apache.org Delivered-To: apmail-hive-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 0F8D5DA4F for ; Mon, 27 Aug 2012 07:58:17 +0000 (UTC) Received: (qmail 18226 invoked by uid 500); 27 Aug 2012 07:58:16 -0000 Delivered-To: apmail-hive-dev-archive@hive.apache.org Received: (qmail 16007 invoked by uid 500); 27 Aug 2012 07:58:09 -0000 Mailing-List: contact dev-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hive.apache.org Delivered-To: mailing list dev@hive.apache.org Received: (qmail 15868 invoked by uid 500); 27 Aug 2012 07:58:08 -0000 Delivered-To: apmail-hadoop-hive-dev@hadoop.apache.org Received: (qmail 15845 invoked by uid 99); 27 Aug 2012 07:58:07 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 27 Aug 2012 07:58:07 +0000 Date: Mon, 27 Aug 2012 18:58:07 +1100 (NCT) From: "Muga Nishizawa (JIRA)" To: hive-dev@hadoop.apache.org Message-ID: <1345649235.838.1346054287593.JavaMail.jiratomcat@arcas> Subject: [jira] [Created] (HIVE-3408) A race condition is caused within QueryPlan class MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 Muga Nishizawa created HIVE-3408: ------------------------------------ Summary: A race condition is caused within QueryPlan class Key: HIVE-3408 URL: https://issues.apache.org/jira/browse/HIVE-3408 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.7.1 Environment: * Java 1.6.0_21, Java HotSpot(TM) 64-Bit Server VM * Ubuntu 10.04 Reporter: Muga Nishizawa Hive's threads are stopped at HashMap.getEntry(..) that is used within QueryPlan#extractCounters() and QueryPlan#updateCountersInQueryPlan(). It seems that a race condition problem of a HashSet object is caused when extractCounters() and updateCountersInQueryPlan() are concurrently executed. I hit the problem with Hive 0.7.1 but, I think that it also is caused with 0.8.1. The following is a part of the thread dump. {quote} "Thread-1091" prio=10 tid=0x00007fd17112b000 nid=0x1100 runnable [0x00007fd175f60000] java.lang.Thread.State: RUNNABLE at java.util.HashMap.getEntry(HashMap.java:347) at java.util.HashMap.containsKey(HashMap.java:335) at java.util.HashSet.contains(HashSet.java:184) at org.apache.hadoop.hive.ql.QueryPlan.extractCounters(QueryPlan.java:342) at org.apache.hadoop.hive.ql.QueryPlan.getQueryPlan(QueryPlan.java:419) at org.apache.hadoop.hive.ql.QueryPlan.toString(QueryPlan.java:592) at org.apache.hadoop.hive.ql.history.HiveHistory.logPlanProgress(HiveHistory.java:493) at org.apache.hadoop.hive.ql.exec.ExecDriver.progress(ExecDriver.java:395) at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:686) at org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:123) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:130) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57) at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:47) "Thread-1090" prio=10 tid=0x00007fd17012f000 nid=0x10ff runnable [0x00007fd175152000] java.lang.Thread.State: RUNNABLE at java.util.HashMap.getEntry(HashMap.java:347) at java.util.HashMap.containsKey(HashMap.java:335) at java.util.HashSet.contains(HashSet.java:184) at org.apache.hadoop.hive.ql.QueryPlan.updateCountersInQueryPlan(QueryPlan.java:297) at org.apache.hadoop.hive.ql.QueryPlan.getQueryPlan(QueryPlan.java:420) at org.apache.hadoop.hive.ql.QueryPlan.toString(QueryPlan.java:592) at org.apache.hadoop.hive.ql.history.HiveHistory.logPlanProgress(HiveHistory.java:493) at org.apache.hadoop.hive.ql.exec.ExecDriver.progress(ExecDriver.java:395) at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:686) at org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:123) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:130) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57) at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:47) {quote} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira