hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konstantin Boudnik (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-1114) Speed up ivy resolution in builds with clever caching
Date Thu, 10 Dec 2009 03:11:18 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12788462#action_12788462
] 

Konstantin Boudnik commented on MAPREDUCE-1114:
-----------------------------------------------

May be I'm barking on a wrong tree but I've tried to run a couple of commands in current MR
trunk:
{noformat}
 % time ant ivy-resolve-common
Buildfile: build.xml
...
BUILD SUCCESSFUL
Total time: 3 seconds

real    0m4.513s
user    0m5.186s
sys     0m0.616s
{noformat}

I got very close result for {{% time ant ivy-resolve-tree}}

so it seems to me that resolver works pretty damn fast considering the number of artifacts
it needs to check. May be the latency is hiding somewhere else?

> Speed up ivy resolution in builds with clever caching
> -----------------------------------------------------
>
>                 Key: MAPREDUCE-1114
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1114
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: build
>    Affects Versions: 0.22.0
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>            Priority: Minor
>         Attachments: mapreduce-1114.txt, mapreduce-1114.txt, mapreduce-1114.txt
>
>
> An awful lot of time is spent in the ivy:resolve parts of the build, even when all of
the dependencies have been fetched and cached. Profiling showed this was in XML parsing. I
have a sort-of-ugly hack which speeds up incremental compiles (and more importantly "ant test")
significantly using some ant macros to cache the resolved classpaths.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message