ant-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gene Smith (JIRA)" <j...@apache.org>
Subject [jira] Updated: (IVY-1252) symlinkmass feature based on symlink feature of ivy:retrieve
Date Thu, 02 Dec 2010 20:10:11 GMT

     [ https://issues.apache.org/jira/browse/IVY-1252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Gene Smith updated IVY-1252:
----------------------------

    Attachment: ivy-docs-update-text.txt
                apache-ivy-2.2.0-src-for-symlinkmass.tar

The source for the feature.  And text which would go into the documentation with it.

> symlinkmass feature based on symlink feature of ivy:retrieve
> ------------------------------------------------------------
>
>                 Key: IVY-1252
>                 URL: https://issues.apache.org/jira/browse/IVY-1252
>             Project: Ivy
>          Issue Type: New Feature
>          Components: Ant, Core, Documentation
>    Affects Versions: 2.2.0
>         Environment: UNIX/Linux
>            Reporter: Gene Smith
>            Priority: Minor
>         Attachments: apache-ivy-2.2.0-src-for-symlinkmass.tar, ivy-docs-update-text.txt
>
>   Original Estimate: 2h
>  Remaining Estimate: 2h
>
> I have an improvement I have made in a locally built copy of Ivy, to speed up the making
of symbolic links when retrieving dependencies.  It was written based on:
>     apache-ivy-2.2.0-src.tar.gz
> It changes existing files, but creates no new files.  I have already made and am using
the change in our local build system, and want to contribute back so that others can use it.
> ....
> The documentation for this feature would be added to:
>         http://ant.apache.org/ivy/history/latest-milestone/use/retrieve.html
> in the main table:
>     Attribute
>         symlinkmass
>     Description
>         true to create symbolic links in mass, false to copy the artifacts. 
>         symlinkmass overrides "symlink" if both are set to "true".
>         symlinkmass will create the same symbolic links "symlink" does, 
>         but with a single process call to "sh" with batched "ln" commands passed in as
standard input.
>         (works on UNIX/Linux, on other systems you may need to script it)
>         Far large lists of resolved jars, this can be dramatically faster.
>         The destination of the symbolic links depends on the value of the useOrigin attribute
>         The events "StartRetrieveArtifactEvent" and EndRetrieveEvent, 
>             are NOT fired by this activity, because it is not clear when they should
be called.
>     Required
>         No. Defaults to false
> ....
> Non trivial code changes are made in:
>     apache-ivy-2.2.0/src/java/org/apache/ivy/core/retrieve/RetrieveEngine.java
>         branching in the code based on the "symlinkmass" boolean 
>             in existing retrieval loop (where fetching of files would normally occur)
to collect destination and source paths 
>             and after the loop to do the mass fetch
>         import added for java.util.HashMap
> A non trivial method is added to handle linking of files based on a Map of destinations
and sources.  Existing methods are untouched.
>     apache-ivy-2.2.0/src/java/org/apache/ivy/util/FileUtil.java
>         a method to do the symlinking based of a Map of the destinations to sources
>         imports added for java.util.Iterator, java.util.LinkedList, java.util.Map, java.util.regex.Pattern
> Trivial additions are made to pass the control parameter "symlinkmass"
>     apache-ivy-2.2.0/src/java/org/apache/ivy/core/retrieve/RetrieveOptions.java
>     apache-ivy-2.2.0/src/java/org/apache/ivy/ant/IvyRetrieve.java
>     apache-ivy-2.2.0/src/java/org/apache/ivy/Main.java
> A unit test is added in
>     apache-ivy-2.2.0/test/java/org/apache/ivy/core/retrieve/RetrieveTest.java
>         basically a duplicate of the unit test for "symlink" with a few internal code
comments
> ...
> Notes on why we want this:
> The build system I maintain (Ant + Ivy, with Linux scripts for Developer support) was
using the "symlink" feature for ivy:retrieve, because over a complete build of our code modules
(about 25 separate and growing) the sub-builds make a total of about 3000 dependency links
and the file space savings made it complete worth it.   The build system is not about speed.
 It is about modularity, and we use Ivy to link the modules in a dependency tree so that we
can swap them out as we .   We like Ivy very much for that.  Thank you all. 
> But, while speed is a secondary consideration, it is not unimportant... and this improvement
cuts about 8 minutes off an 18 minute build.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message