ant-ivy-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Misha Dmitriev <mi...@cloudera.com>
Subject Problem when building the same project concurrently for several OSes
Date Tue, 25 Oct 2016 20:50:03 GMT
Hi everybody,

We have some projects, like Zookeeper, that use Ant/Ivy. In some situations
such a project needs to be built multiple times, for different OSes. To
save time, we do it on the same machine and in parallel (we use docker
containers to build for different OSes). The main problem in this scenario
is when we want all these builds to share the same Ivy cache, i.e. use it
for concurrent reads and writes.

Ivy 2.4 and artifact-lock-nio locking strategy are supposed to address
this. However, when I attempted to switch to this combination and enable a
single cache, my builds failed. I've verified that NIO-based file locking
works across docker containers. I then found and enabled debug logging in
Ivy's FileBasedLockStrategy (one has to invoke ant with
'-Divy.log.locking=true'). It looks like locking for artifacts that are
downloaded from the web works as expected. However, in the end there is a
failure that says "failed to parse report: /home/cauldron/.ivy2/cache/
org.apache.zookeeper-zookeeper-default.xml". See the end of the log file
pasted below. The file in question contains essentially the list of
Zookeeper's external dependencies, and looks like Ivy itself has just
generated it.

So I am confused - why is Ivy trying to read it immediately after writing?
And, generally, is there any chance for us to fix or work around this
problem and use the single, shared Ivy cache?

Regards,

Misha


 [ivy:retrieve] Thread[main,5,main] 1477355454558 entered synchronized area
(locking)
 [ivy:retrieve] Thread[main,5,main] 1477355454558 current status for
/home/cauldron/.ivy2/cache/io.netty/netty/metadatas/metadata-3.10.5.Final.ivy.lck
is 1 held locks: Thread[main,5,main]
 [ivy:retrieve] Thread[main,5,main] 1477355454558 reentrant lock acquired
on /home/cauldron/.ivy2/cache/io.netty/netty/metadatas/metadata-3.10.5.Final.ivy.lck
in 0ms - hold locks = 2
 [ivy:retrieve] Thread[main,5,main] 1477355454558 releasing lock on
/home/cauldron/.ivy2/cache/io.netty/netty/metadatas/
metadata-3.10.5.Final.ivy.lck
 [ivy:retrieve] Thread[main,5,main] 1477355454558 entered synchronized area
(unlocking)
 [ivy:retrieve] Thread[main,5,main] 1477355454558 reentrant lock released
on /home/cauldron/.ivy2/cache/io.netty/netty/metadatas/metadata-3.10.5.Final.ivy.lck
- hold locks = 1
 [ivy:retrieve] Thread[main,5,main] 1477355454559 releasing lock on
/home/cauldron/.ivy2/cache/io.netty/netty/metadatas/
metadata-3.10.5.Final.ivy.lck
 [ivy:retrieve] Thread[main,5,main] 1477355454559 entered synchronized area
(unlocking)
 [ivy:retrieve] Thread[main,5,main] 1477355454559 lock released on
/home/cauldron/.ivy2/cache/io.netty/netty/metadatas/
metadata-3.10.5.Final.ivy.lck
 [ivy:retrieve] :: resolution report :: resolve 468ms :: artifacts dl 131ms
     ---------------------------------------------------------------------
     |                  |            modules            ||   artifacts   |
     |       conf       | number| search|dwnlded|evicted|| number|dwnlded|
     ---------------------------------------------------------------------
     |      default     |   5   |   0   |   0   |   0   ||   5   |   0   |
     ---------------------------------------------------------------------
 [ivy:retrieve] :: retrieving :: org.apache.zookeeper#zookeeper
 [ivy:retrieve]     confs: [default]

 BUILD FAILED
 /container.debian7/output/cdh/zookeeper/zookeeper-3.4.5+cdh6.x+0/build.xml:362:
impossible to ivy retrieve: java.lang.RuntimeException: problem during
retrieve of org.apache.zookeeper#zookeeper: java.text.ParseException:
failed to parse report:
/home/cauldron/.ivy2/cache/org.apache.zookeeper-zookeeper-default.xml:
Premature end of file.
     at org.apache.ivy.core.retrieve.RetrieveEngine.retrieve(
RetrieveEngine.java:249)
     at org.apache.ivy.Ivy.retrieve(Ivy.java:561)
     at org.apache.ivy.ant.IvyRetrieve.doExecute(IvyRetrieve.java:98)
     at org.apache.ivy.ant.IvyTask.execute(IvyTask.java:271)
     at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
     at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(
DelegatingMethodAccessorImpl.java:43)
     at java.lang.reflect.Method.invoke(Method.java:498)
     at org.apache.tools.ant.dispatch.DispatchUtils.execute(
DispatchUtils.java:106)
     at org.apache.tools.ant.Task.perform(Task.java:348)
     at org.apache.tools.ant.Target.execute(Target.java:390)
     at org.apache.tools.ant.Target.performTasks(Target.java:411)
     at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1399)
     at org.apache.tools.ant.Project.executeTarget(Project.java:1368)
     at org.apache.tools.ant.helper.DefaultExecutor.executeTargets(
DefaultExecutor.java:41)
     at org.apache.tools.ant.Project.executeTargets(Project.java:1251)
     at org.apache.tools.ant.Main.runBuild(Main.java:809)
     at org.apache.tools.ant.Main.startAnt(Main.java:217)
     at org.apache.tools.ant.launch.Launcher.run(Launcher.java:280)
     at org.apache.tools.ant.launch.Launcher.main(Launcher.java:109)
 Caused by: java.text.ParseException: failed to parse report:
/home/cauldron/.ivy2/cache/org.apache.zookeeper-zookeeper-default.xml:
Premature end of file.
     at org.apache.ivy.plugins.report.XmlReportParser.parse(
XmlReportParser.java:293)
     at org.apache.ivy.core.retrieve.RetrieveEngine.
determineArtifactsToCopy(RetrieveEngine.java:329)
     at org.apache.ivy.core.retrieve.RetrieveEngine.retrieve(
RetrieveEngine.java:118)
     ... 19 more
 Caused by: org.xml.sax.SAXParseException; systemId:
file:/home/cauldron/.ivy2/cache/org.apache.zookeeper-zookeeper-default.xml;
lineNumber: 1; columnNumber: 1; Premature end of file.
     at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.
createSAXParseException(ErrorHandlerWrapper.java:203)
     at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.
fatalError(ErrorHandlerWrapper.java:177)
     at com.sun.org.apache.xerces.internal.impl.
XMLErrorReporter.reportError(XMLErrorReporter.java:400)
     at com.sun.org.apache.xerces.internal.impl.
XMLErrorReporter.reportError(XMLErrorReporter.java:327)
     at com.sun.org.apache.xerces.internal.impl.XMLScanner.
reportFatalError(XMLScanner.java:1472)
     at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl$
PrologDriver.next(XMLDocumentScannerImpl.java:1014)
     at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(
XMLDocumentScannerImpl.java:602)
     at com.sun.org.apache.xerces.internal.impl.
XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl
.java:505)
     at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(
XML11Configuration.java:841)
     at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(
XML11Configuration.java:770)
     at com.sun.org.apache.xerces.internal.parsers.XMLParser.
parse(XMLParser.java:141)
     at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(
AbstractSAXParser.java:1213)
     at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$
JAXPSAXParser.parse(SAXParserImpl.java:643)
     at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl.
parse(SAXParserImpl.java:327)
     at javax.xml.parsers.SAXParser.parse(SAXParser.java:328)
     at org.apache.ivy.plugins.report.XmlReportParser$
SaxXmlReportParser.parse(XmlReportParser.java:249)
     at org.apache.ivy.plugins.report.XmlReportParser.parse(
XmlReportParser.java:291)
     ... 21 more

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message