From common-issues-return-156720-archive-asf-public=cust-asf.ponee.io@hadoop.apache.org Tue Aug 28 13:38:04 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 4EE2D18067B for ; Tue, 28 Aug 2018 13:38:04 +0200 (CEST) Received: (qmail 67489 invoked by uid 500); 28 Aug 2018 11:38:03 -0000 Mailing-List: contact common-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list common-issues@hadoop.apache.org Received: (qmail 67287 invoked by uid 99); 28 Aug 2018 11:38:03 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 28 Aug 2018 11:38:03 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id DD2A4C6712 for ; Tue, 28 Aug 2018 11:38:02 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -109.501 X-Spam-Level: X-Spam-Status: No, score=-109.501 tagged_above=-999 required=6.31 tests=[ENV_AND_HDR_SPF_MATCH=-0.5, KAM_ASCII_DIVIDERS=0.8, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, USER_IN_DEF_SPF_WL=-7.5, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id ub7k9N3-9LFr for ; Tue, 28 Aug 2018 11:38:02 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id 2A80E5F382 for ; Tue, 28 Aug 2018 11:38:02 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 843A1E261D for ; Tue, 28 Aug 2018 11:38:01 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id DC40126B60 for ; Tue, 28 Aug 2018 11:38:00 +0000 (UTC) Date: Tue, 28 Aug 2018 11:38:00 +0000 (UTC) From: "Steve Loughran (JIRA)" To: common-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (HADOOP-15679) ShutdownHookManager shutdown time needs to be configurable & extended MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HADOOP-15679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15679: ------------------------------------ Target Version/s: 3.0.3, 2.10.0, 2.9.2, 2.8.5 (was: 2.10.0, 2.9.2, 3.0.3, 2.8.5) Status: Patch Available (was: Open) resubmit > ShutdownHookManager shutdown time needs to be configurable & extended > --------------------------------------------------------------------- > > Key: HADOOP-15679 > URL: https://issues.apache.org/jira/browse/HADOOP-15679 > Project: Hadoop Common > Issue Type: Bug > Components: util > Affects Versions: 3.0.0, 2.8.0 > Reporter: Steve Loughran > Assignee: Steve Loughran > Priority: Major > Attachments: HADOOP-15679-001.patch, HADOOP-15679-002.patch, HADOOP-15679-002.patch, HADOOP-15679-003.patch, HADOOP-15679-branch-2-001.patch, HADOOP-15679-branch-2-001.patch > > > HADOOP-12950 added a timeout on shutdowns to avoid problems with hanging shutdowns. But the timeout is too short for applications where a large flush of data is needed on shutdown. > A key example of this is Spark apps which save their history to object stores, where the file close() call triggers an upload of the final local cached block of data (could be 32+MB), and then execute the final mutipart commit. > Proposed > # make the default sleep time 30s, not 10s > # make it configurable with a time duration property (with minimum time of 1s.?) -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org For additional commands, e-mail: common-issues-help@hadoop.apache.org