ant-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mitch Gitman (JIRA)" <>
Subject [jira] [Commented] (IVY-1363) ivy:buildlist task confused by extends feature using two parents
Date Tue, 28 Aug 2012 23:09:09 GMT


Mitch Gitman commented on IVY-1363:

The attached patch addresses two separate sets of bugs:
* One set that was introduced with some changes that were made to my series of patches for
issues IVY-1359, IVY-1363, IVY-1364.
* Another bug that went unaddressed by a solution to IVY-1347 that was a rationale for the
preceding changes.

When Maarten Coene merged from trunk into the 2.3.x branch some otherwise worthwhile refinements
to the work I'd done to have XmlModuleDescriptorParser correctly locate source parent modules
on the filesystem, he removed the following line:            IvyContext.getContext().getSettings().addResolver(parentModuleResolver);

This line was necessary for the resolve-parent-from-source-location mechanism to work in a
normal and predictable way. Without it, I would see error output like the following on a regular
 [ivy:resolve] :: problems summary ::
 [ivy:resolve] :::: ERRORS
 [ivy:resolve]       unknown resolver;1.0
 [ivy:resolve]       unknown resolver;1.0
 [ivy:resolve]       unknown resolver;1.0

At the risk of sounding like a broken record, as I'd noted before with a previous modification
of this feature, it violates any reasonable contract to be reporting errors for non-error
behaviors that happen in the normal course of things. Simple principle: crying wolf about
nonexistent errors is itself an error.

Worse, right after the error gets reported, something insidious happens. Because the caller,
DefaultRepositoryCacheManager, can't find the correct resolver, it goes and either finds or
creates another one:
                    DependencyResolver resolver = settings.getResolver(resolverName);
                    if (resolver == null) {
                        Message.debug("/tresolver not found: " + resolverName
                            + " => trying to use the one configured for " + mrid);
                        resolver = settings.getResolver(depMD.getResolvedModuleRevisionId());

This second resolver, a ChainResolver, is different from the original FileSystemResolver that
was created. So the FileSystemResolver we go to the trouble to create in the method checkParentModuleOnFilesystem
is never actually used to located the parent module.

Therefore, I have restored the line that adds the parent-specific FileSystemResolver to IvySettings.

Now, Maarten told me the reason he removed that line was that, with it still there, he would
still have the StackOverflowError associated with IVY-1347:

I was not able to reproduce this failure, even though I'm using Windows as Maarten is. 

However, I noticed something suspicious about the particular test project that was attached
to the bug. See:

This test case contains two source modules, childtwo and childone. The ivy.xml for childtwo
indicates that its parent is foo#parent. Since no location is specified, the default location,
the ivy.xml one directory above, is used:
    <info organisation="foo" module="childtwo" revision="1.0" status="integration">
        <extends organisation="foo" module="parent" revision="1.0"/>

The problem is, the ivy.xml at that location is the one for the childone module, not parent.
As a result, Ivy is getting tricked into treating childOne as the parent of childTwo when
it's not. 

When you get into XmlModuleDescriptorParser.resolveParentFromModuleInheritanceRepository,
the resolver named module-inheritance-repository-foo#parent;1.0 has for its only ivyPattern

Then, within the resolveParentFromModuleInheritanceRepository call, you get to the following
line in Repository.findResourceUsingPattern:
                String resourceName = IvyPatternHelper.substitute(pattern, mrid, artifact);
At this point, the values are:
pattern = ivy-2.3.x/test/repositories/IVY-1347/childone/ivy.xml
mrid = foo#parent;1.0
artifact = foo#parent;1.0!ivy.xml(ivy)
resourceName = ivy-2.3.x/test/repositories/IVY-1347/childone/ivy.xml

You see the unkosher mixing of parent and childone. And findResourceUsingPattern indeed returns
a ResolvedResource of:
ivy-2.3.x/test/repositories/IVY-1347/childone/ivy.xml (1.0)

Then in BasicResolver.getDependency, the ResolvedModuleResource rmr becomes foo#parent;1.0.

My sense is the only reason this test isn't blowing up is that there's nothing in the parent
to merge. I also believe that Maarten's experiencing the StackOverflowError had something
to do with Ivy trying to resolve this invalid combination.

Now, with the attached patch, Ivy ignores the false positive, and no purported source-located
parent resolver gets added to the Ivy settings. Instead, the parent has to be found in the
regular repository.

To accomplish this, I've added a method isExpectedParentInLocation to XmlModuleDescriptorParser.
If that method returns false, resolveParentFromModuleInheritanceRepository is not called.

The isExpectedParentInLocation implementation relies on a new class, MinimalParentParser,
that I've added as a static class in XmlModuleDescriptorParser. This parser parses just enough
of the parent ivy.xml to find the organisation/module/revision combination. From here, the
Parser in XmlModuleDescriptorParser checks if the ModuleRevisionId in the parent ivy.xml matches
the ModuleRevisionId in the extends element of the child ivy.xml. This includes obtaining
a VersionMatcher and calling accept on it, so that a desired revision like "latest.integration"
in the extends will match something like "1.0" in the parent:
                    VersionMatcher versionMatcher = settings.getVersionMatcher();
                    if (versionMatcher.accept(expectedParentMrid, actualParentMrid)) {
                        return true;

This solution required adding a getVersionMatcher() method to the ParserSettings interface.

NOTE: I've created this patch only for the 2.3.x branch, but not for trunk.
> ivy:buildlist task confused by extends feature using two parents
> ----------------------------------------------------------------
>                 Key: IVY-1363
>                 URL:
>             Project: Ivy
>          Issue Type: Bug
>          Components: Ant, Core
>    Affects Versions: 2.3.0-RC1
>         Environment: Ant 1.7.1 (but should be the same with Ant 1.8.3)
>            Reporter: Mitch Gitman
>            Assignee: Nicolas Lalevée
>              Labels: patch, testcase
>             Fix For: 2.3.0, trunk
>         Attachments:, IVY-1347-1363.patch, IVY-1359-1363-1364-2.3.x.patch,
ivy-2.3.x.patch, ivy-trunk.patch
> I'm finding that the ivy:buildlist Ant task is erroring when it encounters more than
one parent Ivy module that's pulled in through the /ivy-module/info/extends element. This
problem is new to Ivy 2.3.0-rc1; I did not encounter it with Ivy 2.2.0. There is no relationship
or interaction between the two different parent Ivy modules, i.e. no nesting of parents.
> In my test case, which I explain shortly, when I point the ivy:buildlist Ant task at
a project stack that includes a mix of both parents and their children, I see this error:
> ...\multimodule-build\build.xml:28: impossible to parse ivy file for ...\testTwoParents\germany\build.xml:
ivyfile=...\testTwoParents\germany\ivy.xml exception=java.text.ParseException: Problem occurred
while parsing ivy file: inconsistent module descriptor file found in 'file:/.../testTwoParents/master-parent/ivy.xml':
bad module name: expected='bootstrap-parent' found='master-parent';  in file:/.../testTwoParents/germany/ivy.xml
> What's happening is, the germany module extends bootstrap-parent, but somehow the relative
path to master-parent/ivy.xml is supplanting the relative path to bootstrap-parent/ivy.xml.
It appears buildlist doesn't know how to deal with more than one parent.
> This is what occurs when the haltOnError attribute is set to "true" on ivy:buildlist.
If I set haltOnError="false" in my test case, the exception goes away, but I see the following
build order:
> 1. germany
> 2. ireland
> 3. bootstrap-parent
> 4. master-parent
> 5. croatia
> What's wrong about this build order is that germany depends on ireland and ireland depends
on bootstrap-parent, so the order of the first three entries is reversed. If I removed the
dependency of ireland on bootstrap-parent, the order would be the same. This misordering is
clearly related to the presence of more than one parent because comparable tests using (A)
the extends feature with a single parent and (B) no extends feature at all get the order right.
Plus, I see this unexpected output when haltOnError="false":
> [ivy:buildlist] 	=> adding it at the beginning of the path
> [ivy:buildlist] 	=> adding it at the beginning of the path
> I've attached a ZIP containing three standalone test cases, each consisting of a suite
of Ant projects that together comprise a multimodule build whose build order is to be determined
by the ivy:buildlist task:
> * testNoParents: The extends feature is not used.
> * testOneParent: The extends feature is used to pull in content from a single parent
Ivy module.
> * testTwoParents: The extends feature is used where one Ivy module pulls in content from
one parent and two other Ivy modules pull in content from a different parent.
> The testNoParents and testOneParent tests are the control groups. The testTwoParents
test is where things fail.
> When running Ant from one of these test cases, you need to specify the location of the
Ivy 2.3.0-rc1 installation using one of the following:
> * an IVY_DIR environment variable
> * an env.IVY_DIR user property, i.e. -Denv.IVY_DIR=...
> * an ivy.dir user property, i.e. -Divy.dir=...
> To run the build in any of these suites, go to the multimodule-build directory, and execute
"ant" or "ant init"—while specifying the Ivy installation location. You can also run "ant
cleancache" to clear out the Ivy cache. However, you shouldn't need to do this regularly because
each of these test cases uses its own dedicated Ivy cache.
> NOTE: This issue is broached in the email thread "extends & buildlist on 2.3.0-rc1
… it gets worse" on the ivy-user and ant-dev mailing lists.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

View raw message