hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Miklos Szegedi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-6989) [Umbrella] Uploader tool for Distributed Cache Deploy
Date Tue, 02 Jan 2018 19:23:00 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-6989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16308551#comment-16308551
] 

Miklos Szegedi commented on MAPREDUCE-6989:
-------------------------------------------

[~ctrezzo], thanks for the comment. I believe they can extend each other but they have slightly
distinct functionality. This tool is rather a collector for multiple jars in the class path
to a single tarball and it also uploads but that is just a auxiliary task. It could leverage
the shared cache though not to upload a duplicate instance of the same jar. Please let me
know, if I am missing something. Now that you mentioned, there is a need to delete the jar
once no applications are using it. It would be very useful, if we could solve that with the
shared cache.

> [Umbrella] Uploader tool for Distributed Cache Deploy
> -----------------------------------------------------
>
>                 Key: MAPREDUCE-6989
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6989
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>            Reporter: Miklos Szegedi
>            Assignee: Miklos Szegedi
>         Attachments: MAPREDUCE-6989 Mapreduce framework uploader tool.pdf
>
>
> The proposal is to create a tool that collects all available jars in the Hadoop classpath
and adds them to a single tarball file. It then uploads the resulting archive to an HDFS directory.
This saves the cluster administrator from having to set this up manually for Distributed Cache
Deploy.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org


Mime
View raw message