hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Miklos Szegedi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-6989) [Umbrella] Uploader tool for Distributed Cache Deploy
Date Tue, 02 Jan 2018 19:23:00 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-6989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16308551#comment-16308551

Miklos Szegedi commented on MAPREDUCE-6989:

[~ctrezzo], thanks for the comment. I believe they can extend each other but they have slightly
distinct functionality. This tool is rather a collector for multiple jars in the class path
to a single tarball and it also uploads but that is just a auxiliary task. It could leverage
the shared cache though not to upload a duplicate instance of the same jar. Please let me
know, if I am missing something. Now that you mentioned, there is a need to delete the jar
once no applications are using it. It would be very useful, if we could solve that with the
shared cache.

> [Umbrella] Uploader tool for Distributed Cache Deploy
> -----------------------------------------------------
>                 Key: MAPREDUCE-6989
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6989
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>            Reporter: Miklos Szegedi
>            Assignee: Miklos Szegedi
>         Attachments: MAPREDUCE-6989 Mapreduce framework uploader tool.pdf
> The proposal is to create a tool that collects all available jars in the Hadoop classpath
and adds them to a single tarball file. It then uploads the resulting archive to an HDFS directory.
This saves the cluster administrator from having to set this up manually for Distributed Cache

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org

View raw message