From yarn-issues-return-135970-archive-asf-public=cust-asf.ponee.io@hadoop.apache.org Thu Jan 25 04:54:09 2018 Return-Path: X-Original-To: archive-asf-public@eu.ponee.io Delivered-To: archive-asf-public@eu.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by mx-eu-01.ponee.io (Postfix) with ESMTP id AC77318076D for ; Thu, 25 Jan 2018 04:54:09 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 9C51C160C3C; Thu, 25 Jan 2018 03:54:09 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id E4271160C4F for ; Thu, 25 Jan 2018 04:54:08 +0100 (CET) Received: (qmail 52243 invoked by uid 500); 25 Jan 2018 03:54:08 -0000 Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list yarn-issues@hadoop.apache.org Received: (qmail 52230 invoked by uid 99); 25 Jan 2018 03:54:07 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 25 Jan 2018 03:54:07 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 8FDF31A0C47 for ; Thu, 25 Jan 2018 03:54:07 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -100.711 X-Spam-Level: X-Spam-Status: No, score=-100.711 tagged_above=-999 required=6.31 tests=[RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id dF1MkYGtalQu for ; Thu, 25 Jan 2018 03:54:06 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id 02D355FB74 for ; Thu, 25 Jan 2018 03:54:06 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id E3058E045B for ; Thu, 25 Jan 2018 03:54:04 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id DC9932410E for ; Thu, 25 Jan 2018 03:54:02 +0000 (UTC) Date: Thu, 25 Jan 2018 03:54:02 +0000 (UTC) From: "Miklos Szegedi (JIRA)" To: yarn-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (YARN-2185) Use pipes when localizing archives MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/YARN-2185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-2185: --------------------------------- Attachment: YARN-2185.012.patch > Use pipes when localizing archives > ---------------------------------- > > Key: YARN-2185 > URL: https://issues.apache.org/jira/browse/YARN-2185 > Project: Hadoop YARN > Issue Type: Improvement > Components: nodemanager > Affects Versions: 2.4.0 > Reporter: Jason Lowe > Assignee: Miklos Szegedi > Priority: Major > Attachments: YARN-2185.000.patch, YARN-2185.001.patch, YARN-2185.002.patch, YARN-2185.003.patch, YARN-2185.004.patch, YARN-2185.005.patch, YARN-2185.006.patch, YARN-2185.007.patch, YARN-2185.008.patch, YARN-2185.009.patch, YARN-2185.010.patch, YARN-2185.011.patch, YARN-2185.012.patch > > > Currently the nodemanager downloads an archive to a local file, unpacks it, and then removes it. It would be more efficient to stream the data as it's being unpacked to avoid both the extra disk space requirements and the additional disk activity from storing the archive. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org For additional commands, e-mail: yarn-issues-help@hadoop.apache.org