hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2185) Use pipes when localizing archives
Date Wed, 31 Jan 2018 16:09:06 GMT

    [ https://issues.apache.org/jira/browse/YARN-2185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16347067#comment-16347067
] 

Hudson commented on YARN-2185:
------------------------------

SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #13588 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/13588/])
YARN-2185. Use pipes when localizing archives. Contributed by Miklos (jlowe: rev 7288b08330001102f944124d18e02bf0585d3e96)
* (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileUtil.java
* (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/FSDownload.java
* (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/RunJar.java
* (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestFSDownload.java


> Use pipes when localizing archives
> ----------------------------------
>
>                 Key: YARN-2185
>                 URL: https://issues.apache.org/jira/browse/YARN-2185
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: nodemanager
>    Affects Versions: 2.4.0
>            Reporter: Jason Lowe
>            Assignee: Miklos Szegedi
>            Priority: Major
>             Fix For: 3.1.0
>
>         Attachments: YARN-2185.000.patch, YARN-2185.001.patch, YARN-2185.002.patch, YARN-2185.003.patch,
YARN-2185.004.patch, YARN-2185.005.patch, YARN-2185.006.patch, YARN-2185.007.patch, YARN-2185.008.patch,
YARN-2185.009.patch, YARN-2185.010.patch, YARN-2185.011.patch, YARN-2185.012.patch, YARN-2185.012.patch,
YARN-2185.013.patch, YARN-2185.014.patch
>
>
> Currently the nodemanager downloads an archive to a local file, unpacks it, and then
removes it.  It would be more efficient to stream the data as it's being unpacked to avoid
both the extra disk space requirements and the additional disk activity from storing the archive.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message