pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mona Chitnis (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (PIG-3861) duplicate jars get added to distributed cache
Date Thu, 29 May 2014 20:37:02 GMT

     [ https://issues.apache.org/jira/browse/PIG-3861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Mona Chitnis updated PIG-3861:
------------------------------

    Attachment: PIG-3861-4.patch

Patch (v4) updated to resolve issue where pig job submitted via Oozie would report local FS
path for jar instead of on distributed cache. Also rebased to updated trunk. Tested locally

> duplicate jars get added to distributed cache
> ---------------------------------------------
>
>                 Key: PIG-3861
>                 URL: https://issues.apache.org/jira/browse/PIG-3861
>             Project: Pig
>          Issue Type: Bug
>            Reporter: Mona Chitnis
>            Assignee: Mona Chitnis
>            Priority: Minor
>         Attachments: PIG-3681-1.patch, PIG-3861-2.patch, PIG-3861-3.patch, PIG-3861-4.patch
>
>
> PigContext's scriptJars should handle de-duplication of jars to account for script engines
e.g. JythonScriptEngine performing various jar loading for module and sometimes adding same
jar twice. AlsoJobControlCompiler.shipToHdfs() needs a check against adding the same jar more
than once, under different randomly incremented sub-dirs.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message