Return-Path: X-Original-To: apmail-tajo-dev-archive@minotaur.apache.org Delivered-To: apmail-tajo-dev-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 7F009101A0 for ; Thu, 12 Sep 2013 12:34:18 +0000 (UTC) Received: (qmail 16958 invoked by uid 500); 12 Sep 2013 12:34:18 -0000 Delivered-To: apmail-tajo-dev-archive@tajo.apache.org Received: (qmail 16937 invoked by uid 500); 12 Sep 2013 12:34:18 -0000 Mailing-List: contact dev-help@tajo.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@tajo.incubator.apache.org Delivered-To: mailing list dev@tajo.incubator.apache.org Received: (qmail 16928 invoked by uid 99); 12 Sep 2013 12:34:17 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 12 Sep 2013 12:34:17 +0000 X-ASF-Spam-Status: No, hits=-2000.7 required=5.0 tests=ALL_TRUSTED,RP_MATCHES_RCVD X-Spam-Check-By: apache.org Received: from [140.211.11.3] (HELO mail.apache.org) (140.211.11.3) by apache.org (qpsmtpd/0.29) with SMTP; Thu, 12 Sep 2013 12:34:13 +0000 Received: (qmail 16451 invoked by uid 99); 12 Sep 2013 12:33:51 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 12 Sep 2013 12:33:51 +0000 Date: Thu, 12 Sep 2013 12:33:51 +0000 (UTC) From: "Jinho Kim (JIRA)" To: dev@tajo.incubator.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (TAJO-168) infinite loop occurs when QueryMaster is stopping MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/TAJO-168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jinho Kim updated TAJO-168: --------------------------- Attachment: TAJO-168.patch This problem causes the available resource to be deadlock. I've changed the request queue for re-use. > infinite loop occurs when QueryMaster is stopping > ------------------------------------------------- > > Key: TAJO-168 > URL: https://issues.apache.org/jira/browse/TAJO-168 > Project: Tajo > Issue Type: Bug > Components: master, resource manager > Reporter: Hyunsik Choi > Assignee: Jinho Kim > Priority: Blocker > Fix For: 0.2-incubating > > Attachments: TAJO-168.patch > > > See the below log. 'mvn install' sometimes incurs the following infinite loop. > {noformat} > 2013-09-09 11:15:10,249 INFO querymaster.QueryMasterTask (QueryMasterTask.java:stop(153)) - Stopping QueryMasterTask:q_1378692829151_0022 > 2013-09-09 11:15:10,249 INFO querymaster.QueryJobManager (QueryJobManager.java:stopQuery(127)) - Stop QueryInProgress:q_1378692829151_0022 > 2013-09-09 11:15:10,250 INFO master.TajoAsyncDispatcher (TajoAsyncDispatcher.java:stop(122)) - AsyncDispatcher stopped:q_1378692829151_0022 > 2013-09-09 11:15:10,250 INFO querymaster.QueryMasterTask (QueryMasterTask.java:stop(160)) - Stopped QueryMasterTask:q_1378692829151_0022 > 2013-09-09 11:15:10,250 WARN querymaster.QueryMaster (QueryMaster.java:run(325)) - Query q_1378692829151_0018 stopped cause query sesstion timeout: 62808 ms > 2013-09-09 11:15:10,250 WARN querymaster.QueryMaster (QueryMaster.java:run(325)) - Query q_1378692829151_0019 stopped cause query sesstion timeout: 62285 ms > 2013-09-09 11:15:10,250 WARN querymaster.QueryMaster (QueryMaster.java:run(325)) - Query q_1378692829151_0017 stopped cause query sesstion timeout: 63832 ms > 2013-09-09 11:15:11,251 WARN querymaster.QueryMaster (QueryMaster.java:run(325)) - Query q_1378692829151_0021 stopped cause query sesstion timeout: 62251 ms > 2013-09-09 11:15:11,252 WARN querymaster.QueryMaster (QueryMaster.java:run(325)) - Query q_1378692829151_0022 stopped cause query sesstion timeout: 61742 ms > 2013-09-09 11:15:11,252 WARN querymaster.QueryMaster (QueryMaster.java:run(325)) - Query q_1378692829151_0018 stopped cause query sesstion timeout: 63810 ms > 2013-09-09 11:15:11,252 WARN querymaster.QueryMaster (QueryMaster.java:run(325)) - Query q_1378692829151_0019 stopped cause query sesstion timeout: 63287 ms > 2013-09-09 11:15:11,252 WARN querymaster.QueryMaster (QueryMaster.java:run(325)) - Query q_1378692829151_0017 stopped cause query sesstion timeout: 64834 ms > 2013-09-09 11:15:12,253 WARN querymaster.QueryMaster (QueryMaster.java:run(325)) - Query q_1378692829151_0021 stopped cause query sesstion timeout: 63253 ms > 2013-09-09 11:15:12,253 WARN querymaster.QueryMaster (QueryMaster.java:run(325)) - Query q_1378692829151_0022 stopped cause query sesstion timeout: 62743 ms > 2013-09-09 11:15:12,253 WARN querymaster.QueryMaster (QueryMaster.java:run(325)) - Query q_1378692829151_0025 stopped cause query sesstion timeout: 60704 ms > 2013-09-09 11:15:12,254 INFO querymaster.QueryMasterTask (QueryMasterTask.java:stop(153)) - Stopping QueryMasterTask:q_1378692829151_0025 > 2013-09-09 11:15:12,254 INFO querymaster.QueryJobManager (QueryJobManager.java:stopQuery(127)) - Stop QueryInProgress:q_1378692829151_0025 > 2013-09-09 11:15:12,255 INFO master.TajoAsyncDispatcher (TajoAsyncDispatcher.java:stop(122)) - AsyncDispatcher stopped:q_1378692829151_0025 > 2013-09-09 11:15:12,255 INFO querymaster.QueryMasterTask (QueryMasterTask.java:stop(160)) - Stopped QueryMasterTask:q_1378692829151_0025 > 2013-09-09 11:15:12,255 WARN querymaster.QueryMaster (QueryMaster.java:run(325)) - Query q_1378692829151_0026 stopped cause query sesstion timeout: 60197 ms > 2013-09-09 11:15:12,255 INFO querymaster.QueryMasterTask (QueryMasterTask.java:stop(153)) - Stopping QueryMasterTask:q_1378692829151_0026 > 2013-09-09 11:15:12,255 INFO master.TajoAsyncDispatcher (TajoAsyncDispatcher.java:stop(122)) - AsyncDispatcher stopped:q_1378692829151_0026 > 2013-09-09 11:15:12,255 INFO querymaster.QueryMasterTask (QueryMasterTask.java:stop(160)) - Stopped QueryMasterTask:q_1378692829151_0026 > 2013-09-09 11:15:12,255 INFO querymaster.QueryJobManager (QueryJobManager.java:stopQuery(127)) - Stop QueryInProgress:q_1378692829151_0026 > 2013-09-09 11:15:12,256 WARN querymaster.QueryMaster (QueryMaster.java:run(325)) - Query q_1378692829151_0018 stopped cause query sesstion timeout: 64814 ms > 2013-09-09 11:15:12,256 WARN querymaster.QueryMaster (QueryMaster.java:run(325)) - Query q_1378692829151_0019 stopped cause query sesstion timeout: 64291 ms > 2013-09-09 11:15:12,256 WARN querymaster.QueryMaster (QueryMaster.java:run(325)) - Query q_1378692829151_0017 stopped cause query sesstion timeout: 65838 ms > 2013-09-09 11:15:13,257 WARN querymaster.QueryMaster (QueryMaster.java:run(325)) - Query q_1378692829151_0021 stopped cause query sesstion timeout: 64257 ms > 2013-09-09 11:15:13,257 WARN querymaster.QueryMaster (QueryMaster.java:run(325)) - Query q_1378692829151_0022 stopped cause query sesstion timeout: 63747 ms > 2013-09-09 11:15:13,257 WARN querymaster.QueryMaster (QueryMaster.java:run(325)) - Query q_1378692829151_0025 stopped cause query sesstion timeout: 61708 ms > 2013-09-09 11:15:13,257 WARN querymaster.QueryMaster (QueryMaster.java:run(325)) - Query q_1378692829151_0026 stopped cause query sesstion timeout: 61199 ms > 2013-09-09 11:15:13,257 WARN querymaster.QueryMaster (QueryMaster.java:run(325)) - Query q_1378692829151_0027 stopped cause query sesstion timeout: 60685 ms > 2013-09-09 11:15:13,257 INFO querymaster.QueryMasterTask (QueryMasterTask.java:stop(153)) - Stopping QueryMasterTask:q_1378692829151_0027 > 2013-09-09 11:15:13,258 INFO querymaster.QueryJobManager (QueryJobManager.java:stopQuery(127)) - Stop QueryInProgress:q_1378692829151_0027 > 2013-09-09 11:15:13,258 INFO master.TajoAsyncDispatcher (TajoAsyncDispatcher.java:stop(122)) - AsyncDispatcher stopped:q_1378692829151_0027 > 2013-09-09 11:15:13,258 INFO querymaster.QueryMasterTask (QueryMasterTask.java:stop(160)) - Stopped QueryMasterTask:q_1378692829151_0027 > 2013-09-09 11:15:13,258 WARN querymaster.QueryMaster (QueryMaster.java:run(325)) - Query q_1378692829151_0018 stopped cause query sesstion timeout: 65816 ms > 2013-09-09 11:15:13,259 WARN querymaster.QueryMaster (QueryMaster.java:run(325)) - Query q_1378692829151_0019 stopped cause query sesstion timeout: 65294 ms > 2013-09-09 11:15:13,259 WARN querymaster.QueryMaster (QueryMaster.java:run(325)) - Query q_1378692829151_0017 stopped cause query sesstion timeout: 66841 ms > 2013-09-09 11:15:14,260 WARN querymaster.QueryMaster (QueryMaster.java:run(325)) - Query q_1378692829151_0021 stopped cause query sesstion timeout: 65260 ms > 2013-09-09 11:15:14,260 WARN querymaster.QueryMaster (QueryMaster.java:run(325)) - Query q_1378692829151_0022 stopped cause query sesstion timeout: 64750 ms > 2013-09-09 11:15:14,260 WARN querymaster.QueryMaster (QueryMaster.java:run(325)) - Query q_1378692829151_0025 stopped cause query sesstion timeout: 62711 ms > 2013-09-09 11:15:14,260 WARN querymaster.QueryMaster (QueryMaster.java:run(325)) - Query q_1378692829151_0026 stopped cause query sesstion timeout: 62202 ms > 2013-09-09 11:15:14,260 WARN querymaster.QueryMaster (QueryMaster.java:run(325)) - Query q_1378692829151_0027 stopped cause query sesstion timeout: 61688 ms > 2013-09-09 11:15:14,260 WARN querymaster.QueryMaster (QueryMaster.java:run(325)) - Query q_1378692829151_0018 stopped cause query sesstion timeout: 66818 ms > 2013-09-09 11:15:14,260 WARN querymaster.QueryMaster (QueryMaster.java:run(325)) - Query q_1378692829151_0019 stopped cause query sesstion timeout: 66295 ms > 2013-09-09 11:15:14,260 WARN querymaster.QueryMaster (QueryMaster.java:run(325)) - Query q_1378692829151_0017 stopped cause query sesstion timeout: 67842 ms > 2013-09-09 11:15:15,261 WARN querymaster.QueryMaster (QueryMaster.java:run(325)) - Query q_1378692829151_0021 stopped cause query sesstion timeout: 66261 ms > 2013-09-09 11:15:15,262 WARN querymaster.QueryMaster (QueryMaster.java:run(325)) - Query q_1378692829151_0022 stopped cause query sesstion timeout: 65752 ms > 2013-09-09 11:15:15,262 WARN querymaster.QueryMaster (QueryMaster.java:run(325)) - Query q_1378692829151_0025 stopped cause query sesstion timeout: 63713 ms > 2013-09-09 11:15:15,262 WARN querymaster.QueryMaster (QueryMaster.java:run(325)) - Query q_1378692829151_0026 stopped cause query sesstion timeout: 63204 ms > 2013-09-09 11:15:15,262 WARN querymaster.QueryMaster (QueryMaster.java:run(325)) - Query q_1378692829151_0027 stopped cause query sesstion timeout: 62690 ms > 2013-09-09 11:15:15,262 WARN querymaster.QueryMaster (QueryMaster.java:run(325)) - Query q_1378692829151_0018 stopped cause query sesstion timeout: 67820 ms > 2013-09-09 11:15:15,262 WARN querymaster.QueryMaster (QueryMaster.java:run(325)) - Query q_1378692829151_0019 stopped cause query sesstion timeout: 67297 ms > 2013-09-09 11:15:15,262 WARN querymaster.QueryMaster (QueryMaster.java:run(325)) - Query q_1378692829151_0017 stopped cause query sesstion timeout: 68844 ms > 2013-09-09 11:15:16,263 WARN querymaster.QueryMaster (QueryMaster.java:run(325)) - Query q_1378692829151_0021 stopped cause query sesstion timeout: 67263 ms > 2013-09-09 11:15:16,263 WARN querymaster.QueryMaster (QueryMaster.java:run(325)) - Query q_1378692829151_0022 stopped cause query sesstion timeout: 66753 ms > 2013-09-09 11:15:16,263 WARN querymaster.QueryMaster (QueryMaster.java:run(325)) - Query q_1378692829151_0025 stopped cause query sesstion timeout: 64714 ms > 2013-09-09 11:15:16,263 WARN querymaster.QueryMaster (QueryMaster.java:run(325)) - Query q_1378692829151_0026 stopped cause query sesstion timeout: 64205 ms > 2013-09-09 11:15:16,264 WARN querymaster.QueryMaster (QueryMaster.java:run(325)) - Query q_1378692829151_0027 stopped cause query sesstion timeout: 63692 ms > 2013-09-09 11:15:16,264 WARN querymaster.QueryMaster (QueryMaster.java:run(325)) - Query q_1378692829151_0018 stopped cause query sesstion timeout: 68822 ms > 2013-09-09 11:15:16,264 WARN querymaster.QueryMaster (QueryMaster.java:run(325)) - Query q_1378692829151_0019 stopped cause query sesstion timeout: 68299 ms > 2013-09-09 11:15:16,264 WARN querymaster.QueryMaster (QueryMaster.java:run(325)) - Query q_1378692829151_0017 stopped cause query sesstion timeout: 69846 ms > 2013-09-09 11:15:17,265 WARN querymaster.QueryMaster (QueryMaster.java:run(325)) - Query q_1378692829151_0021 stopped cause query sesstion timeout: 68265 ms > 2013-09-09 11:15:17,266 WARN querymaster.QueryMaster (QueryMaster.java:run(325)) - Query q_1378692829151_0022 stopped cause query sesstion timeout: 67756 ms > 2013-09-09 11:15:17,266 WARN querymaster.QueryMaster (QueryMaster.java:run(325)) - Query q_1378692829151_0025 stopped cause query sesstion timeout: 65717 ms > 2013-09-09 11:15:17,266 WARN querymaster.QueryMaster (QueryMaster.java:run(325)) - Query q_1378692829151_0026 stopped cause query sesstion timeout: 65208 ms > 2013-09-09 11:15:17,266 WARN querymaster.QueryMaster (QueryMaster.java:run(325)) - Query q_1378692829151_0027 stopped cause query sesstion timeout: 64694 ms > 2013-09-09 11:15:17,266 WARN querymaster.QueryMaster (QueryMaster.java:run(325)) - Query q_1378692829151_0018 stopped cause query sesstion timeout: 69824 ms > 2013-09-09 11:15:17,266 WARN querymaster.QueryMaster (QueryMaster.java:run(325)) - Query q_1378692829151_0019 stopped cause query sesstion timeout: 69301 ms > 2013-09-09 11:15:17,266 WARN querymaster.QueryMaster (QueryMaster.java:run(325)) - Query q_1378692829151_0017 stopped cause query sesstion timeout: 70848 ms > {noformat} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira