falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shwetha G S (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (FALCON-943) process update copying user lib is very slow
Date Tue, 16 Dec 2014 10:41:13 GMT

     [ https://issues.apache.org/jira/browse/FALCON-943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Shwetha G S updated FALCON-943:
    Attachment: FALCON-943.patch

Updated patch. Addresses the following:
1. Removed copying user workflow/lib to staging directory
2. Removed update at specific time
3. Copying falcon jars to parent workflow lib copies only specific falcon jars
4. Libs copied at cluster submit(3rd party jars) have versions so that its easier to debug

> process update copying user lib is very slow
> --------------------------------------------
>                 Key: FALCON-943
>                 URL: https://issues.apache.org/jira/browse/FALCON-943
>             Project: Falcon
>          Issue Type: Bug
>            Reporter: Shwetha G S
>            Assignee: Shwetha G S
>         Attachments: FALCON-943.patch, FALCON-943.patch
> Currently, process update copies the user workflow and lib directories to staging directory
to provide workflow versions. If the library contains many jars, hdfs copy takes longer and
the update API becomes very slow. Worse if the feed update in turn calls process update, feed
update takes ~30 mins!

This message was sent by Atlassian JIRA

View raw message