From commits-return-65586-archive-asf-public=cust-asf.ponee.io@beam.apache.org Tue Apr 10 01:46:08 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 5DC4018067B for ; Tue, 10 Apr 2018 01:46:08 +0200 (CEST) Received: (qmail 79036 invoked by uid 500); 9 Apr 2018 23:46:07 -0000 Mailing-List: contact commits-help@beam.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@beam.apache.org Delivered-To: mailing list commits@beam.apache.org Received: (qmail 79027 invoked by uid 99); 9 Apr 2018 23:46:07 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 09 Apr 2018 23:46:07 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 10A07C7A00 for ; Mon, 9 Apr 2018 23:46:07 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -109.501 X-Spam-Level: X-Spam-Status: No, score=-109.501 tagged_above=-999 required=6.31 tests=[ENV_AND_HDR_SPF_MATCH=-0.5, KAM_ASCII_DIVIDERS=0.8, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, USER_IN_DEF_SPF_WL=-7.5, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id 2OQkOdPwYy2D for ; Mon, 9 Apr 2018 23:46:06 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id 2336B5F18F for ; Mon, 9 Apr 2018 23:46:06 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 9C7EFE0B76 for ; Mon, 9 Apr 2018 23:46:00 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 2BA2C241C7 for ; Mon, 9 Apr 2018 23:46:00 +0000 (UTC) Date: Mon, 9 Apr 2018 23:46:00 +0000 (UTC) From: "ASF GitHub Bot (JIRA)" To: commits@beam.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Work logged] (BEAM-3119) direct-metrics-counter-committer threads are leaking MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/BEAM-3119?focusedWorklogId=89198&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-89198 ] ASF GitHub Bot logged work on BEAM-3119: ---------------------------------------- Author: ASF GitHub Bot Created on: 09/Apr/18 23:45 Start Date: 09/Apr/18 23:45 Worklog Time Spent: 10m Work Description: tgroh commented on a change in pull request #4965: BEAM-3119 ensure the metrics thread pool is related to an execution URL: https://github.com/apache/beam/pull/4965#discussion_r180260921 ########## File path: runners/direct-java/src/main/java/org/apache/beam/runners/direct/DirectRunner.java ########## @@ -172,48 +176,53 @@ public DirectPipelineResult run(Pipeline originalPipeline) { } pipeline.replaceAll(defaultTransformOverrides()); MetricsEnvironment.setMetricsSupported(true); Review comment: This probably should restore the previous state, whatever it happened to be, rather than blind-setting to false; However, I'm not super worried about it, because this is already global state so multiple pipelines can interfere with each other. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: users@infra.apache.org Issue Time Tracking ------------------- Worklog Id: (was: 89198) Time Spent: 2h 10m (was: 2h) > direct-metrics-counter-committer threads are leaking > ---------------------------------------------------- > > Key: BEAM-3119 > URL: https://issues.apache.org/jira/browse/BEAM-3119 > Project: Beam > Issue Type: Bug > Components: runner-direct > Reporter: Etienne Chauchot > Assignee: Thomas Groh > Priority: Major > Time Spent: 2h 10m > Remaining Estimate: 0h > > When I run ElasticsearchIOTests using ESv5, there is a thread leak control mechanism ({{com.carrotsearch.randomizedtesting.ThreadLeakControl}}). It waits for 5s for non-terminated threads at the end of a test. It detects leaked {{direct-metrics-counter-committer}} thread. > {code} > com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated: > 1) Thread[id=296, name=direct-metrics-counter-committer, state=TIMED_WAITING, group=TGRP-ElasticsearchIOTest] > at sun.misc.Unsafe.park(Native Method) > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) > at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) > at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) > at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > at __randomizedtesting.SeedInfo.seed([59E504CA1B0DD6A8]:0){code} > I tried to increase the timeout to 30s (by patching randomizedtesting-runner-2.5.0.jar) but still gets a zombie thread. > To reproduce, just comment > {code} > @ThreadLeakScope(ThreadLeakScope.Scope.NONE) > {code} > in > {code} > beam/sdks/java/io/elasticsearch-tests/elasticsearch-tests-5/src/test/java/org/apache/beam/sdk/io/elasticsearch/ElasticsearchIOTest.java > {code} > and run > {code} > testRead() > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)