hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-1114) Speed up ivy resolution in builds with clever caching
Date Fri, 04 Dec 2009 22:43:20 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12786211#action_12786211
] 

Todd Lipcon commented on MAPREDUCE-1114:
----------------------------------------

bq. I don't think the 15 second payoff justifies the maintenance cost of a custom caching
layer for ivy.

Comparing the 15 second payoff to the full build time isn't particular important to me. For
me, the ability to quickly iterate on code while recompiling and rerunning unit tests is the
big payoff - so I look at this as a 60% speedup in my development cycle rather than a few
% speedup in the full build.

I may be in the minority, though, as I don't use eclipse or anything other fancy IDE that
does incremental compilation.

Anyone else care to chime in?

> Speed up ivy resolution in builds with clever caching
> -----------------------------------------------------
>
>                 Key: MAPREDUCE-1114
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1114
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: build
>    Affects Versions: 0.22.0
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>            Priority: Minor
>         Attachments: mapreduce-1114.txt, mapreduce-1114.txt, mapreduce-1114.txt
>
>
> An awful lot of time is spent in the ivy:resolve parts of the build, even when all of
the dependencies have been fetched and cached. Profiling showed this was in XML parsing. I
have a sort-of-ugly hack which speeds up incremental compiles (and more importantly "ant test")
significantly using some ant macros to cache the resolved classpaths.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message