maven-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Claude Warren (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MASSEMBLY-774) too many open files
Date Thu, 28 Jul 2016 17:25:20 GMT

    [ https://issues.apache.org/jira/browse/MASSEMBLY-774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15397856#comment-15397856
] 

Claude Warren commented on MASSEMBLY-774:
-----------------------------------------

I am attempting a build under Jenkins/Linux we set the number of files to 4096
We are using the latest assembly plugin (2.6) as well as the latest plexus-archive plugin
(3.4) and yet I still get a too many open files error.  We eventually upped the number of
file to 20K and got past it.

We are packaging up a dojo implementation so there are lots of files, but I would still expect
it to work.

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-assembly-plugin:2.6:single (make-assembly)
on project dashboard-dojo: Failed to create assembly: Error creating assembly archive sources:
Problem creating zip: /opt/jenkins/workspace/NOI-Solutions/noi-dashboard/dashboard-dojo/target/dojo/dojo/release/dojo/cldr/nls/fil/number.js.uncompressed.js
(Too many open files) -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-assembly-plugin:2.6:single
(make-assembly) on project dashboard-dojo: Failed to create assembly: Error creating assembly
archive sources: Problem creating zip: /opt/jenkins/workspace/NOI-Solutions/noi-dashboard/dashboard-dojo/target/dojo/dojo/release/dojo/cldr/nls/fil/number.js.uncompressed.js
(Too many open files)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
	at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
	at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
	at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
	at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
	at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
	at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:95)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:56)
	at java.lang.reflect.Method.invoke(Method.java:620)
	at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
	at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
	at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
	at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.plugin.MojoExecutionException: Failed to create assembly: Error
creating assembly archive sources: Problem creating zip: /opt/jenkins/workspace/NOI-Solutions/noi-dashboard/dashboard-dojo/target/dojo/dojo/release/dojo/cldr/nls/fil/number.js.uncompressed.js
(Too many open files)
	at org.apache.maven.plugin.assembly.mojos.AbstractAssemblyMojo.execute(AbstractAssemblyMojo.java:541)
	at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
	... 20 more
Caused by: org.apache.maven.plugin.assembly.archive.ArchiveCreationException: Error creating
assembly archive sources: Problem creating zip: /opt/jenkins/workspace/NOI-Solutions/noi-dashboard/dashboard-dojo/target/dojo/dojo/release/dojo/cldr/nls/fil/number.js.uncompressed.js
(Too many open files)
	at org.apache.maven.plugin.assembly.archive.DefaultAssemblyArchiver.createArchive(DefaultAssemblyArchiver.java:185)
	at org.apache.maven.plugin.assembly.mojos.AbstractAssemblyMojo.execute(AbstractAssemblyMojo.java:484)
	... 22 more
Caused by: org.codehaus.plexus.archiver.ArchiverException: Problem creating zip: /opt/jenkins/workspace/NOI-Solutions/noi-dashboard/dashboard-dojo/target/dojo/dojo/release/dojo/cldr/nls/fil/number.js.uncompressed.js
(Too many open files)
	at org.codehaus.plexus.archiver.AbstractArchiver.createArchive(AbstractArchiver.java:933)
	at org.apache.maven.plugin.assembly.archive.archiver.AssemblyProxyArchiver.createArchive(AssemblyProxyArchiver.java:445)
	at org.apache.maven.plugin.assembly.archive.DefaultAssemblyArchiver.createArchive(DefaultAssemblyArchiver.java:181)
	... 23 more
Caused by: java.io.FileNotFoundException: /opt/jenkins/workspace/NOI-Solutions/noi-dashboard/dashboard-dojo/target/dojo/dojo/release/dojo/cldr/nls/fil/number.js.uncompressed.js
(Too many open files)
	at java.io.FileInputStream.open(Native Method)
	at java.io.FileInputStream.<init>(FileInputStream.java:158)
	at org.codehaus.plexus.components.io.resources.PlexusIoFileResource$1.getContents(PlexusIoFileResource.java:99)
	at org.codehaus.plexus.components.io.resources.PlexusIoFileResource.getContents(PlexusIoFileResource.java:124)
	at org.codehaus.plexus.archiver.ArchiveEntry.getInputStream(ArchiveEntry.java:126)
	at org.codehaus.plexus.archiver.zip.AbstractZipArchiver.zipFile(AbstractZipArchiver.java:492)
	at org.codehaus.plexus.archiver.zip.AbstractZipArchiver.addResources(AbstractZipArchiver.java:365)
	at org.codehaus.plexus.archiver.zip.AbstractZipArchiver.createArchiveMain(AbstractZipArchiver.java:321)
	at org.codehaus.plexus.archiver.zip.AbstractZipArchiver.execute(AbstractZipArchiver.java:222)
	at org.codehaus.plexus.archiver.AbstractArchiver.createArchive(AbstractArchiver.java:916)

> too many open files
> -------------------
>
>                 Key: MASSEMBLY-774
>                 URL: https://issues.apache.org/jira/browse/MASSEMBLY-774
>             Project: Maven Assembly Plugin
>          Issue Type: Bug
>    Affects Versions: 2.5.5
>            Reporter: Dan Armbrust
>            Assignee: Kristian Rosenvold
>
> I ran across this - http://mail-archives.apache.org/mod_mbox/maven-dev/201501.mbox/%3CA2EE0D04-04E3-4C99-8842-67346386200C@takari.io%3E
- and since I'm making huge zip files, I thought I would give it a try.
> I configured as:
>                 <plugin>
>                     <groupId>org.apache.maven.plugins</groupId>
>                     <artifactId>maven-assembly-plugin</artifactId>
>                     <version>2.5.5</version>
>                     <dependencies>
>                         <dependency>
>                             <groupId>org.codehaus.plexus</groupId>
>                             <artifactId>plexus-archiver</artifactId>
>                             <version>3.0.1</version>
>                         </dependency>
>                         <dependency>
>                             <groupId>org.codehaus.plexus</groupId>
>                             <artifactId>plexus-io</artifactId>
>                             <version>2.6</version>
>                         </dependency>
>                     </dependencies>
>                 </plugin>
> But this lead to a failure:
> [ERROR] Failed to execute goal org.apache.maven.plugins:maven-assembly-plugin:2.5.5:single
(zip) on project ... sememe/seg.3182.sememe.map (Too many open files) -> [Help 1]
> I certainly could raise my ulimit:
> darmbrust@overkill:/mnt/STORAGE/Work/Apelon/Workspaces/ISAAC-Core-2/va-solor-goods$ ulimit
-a
> ...
> open files                      (-n) 1024
> But it seems it would be better if the zip process limited itself to a reasonable number
of open files.  I don't know the algorithm... but it seems that double or triple the processor
count would be more than enough.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message