harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tony Wu" <wuyue...@gmail.com>
Subject Re: [jira] Created: (HARMONY-2914) [classlib][archive] RFE: java.util.zip.ZipFile(File file) wastes file handlers on Linux
Date Wed, 24 Jan 2007 15:33:04 GMT
IIRC, the ACQ we have signed made some restrictions on touching the
source code of RI. I thought the activities which may be considered as
"accessing the source code" should be prevented, otherwise one would
disqualify from committing to the repository here at Apache Harmony.

If you think that it is ok to do that, it would be ok for me also.

On 1/24/07, Geir Magnusson Jr. <geir@pobox.com> wrote:
>
> On Jan 24, 2007, at 3:36 AM, Tony Wu wrote:
>
> > I think it is implementation detail. Harmony holds the handler when
> > the ZipFile is created. For performance concern, it is ok to follow
> > RI. What's your opinion?
> >
> > And I would argue that it is not very proper to use something like
> > strace to detect the detail behavior of RI.
>
> Why?
>
> >
> > On 12/28/06, Nina Rinskaya (JIRA) <jira@apache.org> wrote:
> >> [classlib][archive] RFE: java.util.zip.ZipFile(File file) wastes
> >> file handlers on Linux
> >> ---------------------------------------------------------------------
> >> ------------------
> >>
> >>                 Key: HARMONY-2914
> >>                 URL: http://issues.apache.org/jira/browse/
> >> HARMONY-2914
> >>             Project: Harmony
> >>          Issue Type: Wish
> >>          Components: Classlib
> >>         Environment: Linux (SLES9), ia32
> >>            Reporter: Nina Rinskaya
> >>            Priority: Minor
> >>
> >>
> >> During Eclipse Unit Tests runs evaluation I've encountered an
> >> issue that looks like file handlers waste/leak on Linux when
> >> opening new zip (jar) file (using java.util.zip.Zipfile(File file)
> >> constructor): unlike RI (Sun), Harmony takes and doesn't releases
> >> file handlers when opening zip (jar) files. When working with
> >> large application with thousands of plugins and jar files in the
> >> classpath, the OS file handlers limit can be reached very quickly
> >> as it happens with some Eclipse Unit Tests on Harmony.
> >>
> >> The issue can be workarounded (and I did it in EUT) by calling GC
> >> each time we expect many zip/jar files opened.
> >>
> >> Note, JRockit 1.5 behaves almost in the same way as Harmony (goes
> >> out of file handlers and starts throwing IOException), but Sun
> >> works ok. I don't know how they did it, but it would be great if
> >> we make it work too.
> >>
> >> Here is the simple way to reproduce the issue:
> >>
> >> 1. Create 'tmp' subdirectory in the working directory, and copy
> >> any jar file 100 times under '$i.jar' names into 'tmp' subdirectory.
> >> 2. Set low open files limit: ulimit -n 60 (reproducible with any
> >> limit actually)
> >> 3. Compile and run the following test (FileTest.java):
> >>
> >> --------------------------------
> >> import java.io.*;
> >> import java.util.zip.ZipFile;
> >>
> >> public class FileTest {
> >>    public static Object[] oo = new Object[100];
> >>    public void testB() {
> >>        for (int i=0; i<100; i++) {
> >>            try {
> >>                oo[i] = new ZipFile(new File("tmp/"+i+".jar"));
> >>            } catch (IOException ioe) {
> >>                ioe.printStackTrace();
> >>                System.out.println("Failed on iteration " + i);
> >>                return;
> >>            }
> >>        }
> >>        System.out.println("OK");
> >>    }
> >>    public static void main(String[] args) {
> >>        FileTest ft = new FileTest();
> >>        ft.testB();
> >>    }
> >> }
> >>
> >> ---------------------------------
> >>
> >> Output on Harmony (looks like SIGABRT is caused by some VM actions
> >> during exit with open files limit reached - can also be a problem) :
> >>
> >> java.util.zip.ZipException: archive.24
> >>        at java.util.zip.ZipFile.openZip(ZipFile.java:120)
> >>        at java.util.zip.ZipFile.<init>(ZipFile.java:111)
> >>        at java.util.zip.ZipFile.<init>(ZipFile.java:71)
> >>        at FileTest2.testB(FileTest2.java:9)
> >>        at FileTest2.main(FileTest2.java:18)
> >> Failed on iteration 12
> >> java: /export/users/nrinskay/harmony_build/trunk/working_vm/vm/
> >> vmcore/src/util/linux/signals_ia32.cpp:254: void* find_stack_addr
> >> (): Assertion `!err' failed.
> >> SIGABRT in VM code.
> >> Stack trace:
> >> Warning: Cannot get modules info, no symbolic information will be
> >> provided
> >>        1: ?? (??:-1)
> >> <end of stack trace>
> >> Aborted
> >> ----------------------------
> >>
> >> Output on JRockit 1.5:
> >>
> >> java.io.IOException: Error opening file
> >>        at java.util.zip.ZipFile.open(Ljava.lang.String;I)I(Unknown
> >> Source)
> >>        at java.util.zip.ZipFile.<init>(Ljava.io.File;I)V(Unknown
> >> Source)
> >>        at java.util.zip.ZipFile.<init>(Ljava.io.File;)V(Unknown
> >> Source)
> >>        at FileTest2.testB()V(FileTest2.java:9)
> >>        at FileTest2.main([Ljava.lang.String;)V(FileTest2.java:20)
> >> Failed on iteration 45
> >> ------------------------------------
> >>
> >> Output on Sun 1.5:
> >>
> >> OK
> >> ------------------------
> >>
> >>
> >> I used strace to compare system calls logs for Sun 1.5, JRockit
> >> 1.5 and Harmony (see below). I'm not sure how it works, but Sun
> >> releases open files handles just after opening zip/jar files and
> >> never goes out of the limit.
> >>
> >> Sun:
> >> -----------------------------
> >> stat64("tmp/0.jar", {st_mode=S_IFREG|0644, st_size=2676, ...}) = 0
> >> open("tmp/0.jar", O_RDONLY|O_LARGEFILE) = 3
> >> fstat64(3, {st_mode=S_IFREG|0644, st_size=2676, ...}) = 0
> >> _llseek(3, 0, [2676], SEEK_END)         = 0
> >> mmap2(NULL, 2676, PROT_READ, MAP_SHARED, 3, 0) = 0x89e9a000
> >> close(3)                                = 0
> >> stat64("tmp/1.jar", {st_mode=S_IFREG|0644, st_size=2676, ...}) = 0
> >> open("tmp/1.jar", O_RDONLY|O_LARGEFILE) = 3
> >> fstat64(3, {st_mode=S_IFREG|0644, st_size=2676, ...}) = 0
> >> _llseek(3, 0, [2676], SEEK_END)         = 0
> >> mmap2(NULL, 2676, PROT_READ, MAP_SHARED, 3, 0) = 0x89e9b000
> >> close(3)                                = 0
> >> stat64("tmp/2.jar", {st_mode=S_IFREG|0644, st_size=2676, ...}) = 0
> >> open("tmp/2.jar", O_RDONLY|O_LARGEFILE) = 3
> >> fstat64(3, {st_mode=S_IFREG|0644, st_size=2676, ...}) = 0
> >> _llseek(3, 0, [2676], SEEK_END)         = 0
> >> mmap2(NULL, 2676, PROT_READ, MAP_SHARED, 3, 0) = 0x89e9c000
> >> close(3)
> >> -----------------------------
> >>
> >> We see "close" call for each opened file.
> >>
> >> Jrockit:
> >> --------------------------
> >> open("tmp/0.jar", O_RDONLY|O_LARGEFILE) = 15
> >> fstat64(15, {st_mode=S_IFREG|0644, st_size=2676, ...}) = 0
> >> pread(15, "PK\5\6\0\0\0\0\5\0\5\0005\1\0\0)\t\0\0\0\0", 22, 2654)
> >> = 22
> >> pread(15, "PK\1\2\24\0\24\0\10\0\10\0\23\230\2325\0\0\0\0\2\0\0
> >> \0"..., 309, 2345) = 309
> >> stat64("tmp/1.jar", {st_mode=S_IFREG|0644, st_size=2676, ...}) = 0
> >> open("tmp/1.jar", O_RDONLY|O_LARGEFILE) = 16
> >> fstat64(16, {st_mode=S_IFREG|0644, st_size=2676, ...}) = 0
> >> pread(16, "PK\5\6\0\0\0\0\5\0\5\0005\1\0\0)\t\0\0\0\0", 22, 2654)
> >> = 22
> >> pread(16, "PK\1\2\24\0\24\0\10\0\10\0\23\230\2325\0\0\0\0\2\0\0
> >> \0"..., 309, 2345) = 309
> >> stat64("tmp/2.jar", {st_mode=S_IFREG|0644, st_size=2676, ...}) = 0
> >> open("tmp/2.jar", O_RDONLY|O_LARGEFILE) = 17
> >> fstat64(17, {st_mode=S_IFREG|0644, st_size=2676, ...}) = 0
> >> pread(17, "PK\5\6\0\0\0\0\5\0\5\0005\1\0\0)\t\0\0\0\0", 22, 2654)
> >> = 22
> >> pread(17, "PK\1\2\24\0\24\0\10\0\10\0\23\230\2325\0\0\0\0\2\0\0
> >> \0"..., 309, 2345) = 309
> >> stat64("tmp/3.jar", {st_mode=S_IFREG|0644, st_size=2676, ...}) = 0
> >> open("tmp/3.jar", O_RDONLY|O_LARGEFILE) = 18
> >> ------------------------------
> >>
> >> - no "close" call.
> >>
> >> Harmony:
> >> ------------------------------
> >> open("tmp/0.jar", O_RDONLY)             = 48
> >> fcntl64(48, F_GETFD)                    = 0
> >> fcntl64(48, F_SETFD, FD_CLOEXEC)        = 0
> >> read(48, "PK\3\4", 4)                   = 4
> >> lseek(48, 0, SEEK_SET)                  = 0
> >> stat64("tmp/0.jar", {st_mode=S_IFREG|0644, st_size=2676, ...}) = 0
> >> stat64("tmp/0.jar", {st_mode=S_IFREG|0644, st_size=2676, ...}) = 0
> >> lseek(48, 0, SEEK_END)                  = 2676
> >> lseek(48, 1652, SEEK_SET)               = 1652
> >> read(48, "hH\304D|\254\340\223*g\2*\346u\3339\301\35>\5
> >> \265Z2-7v"..., 1024) = 1024
> >> lseek(48, 2345, SEEK_SET)               = 2345
> >> read(48, "PK\1\2\24\0\24\0\10\0\10\0\23\230\2325\0\0\0\0\2\0\0
> >> \0"..., 313) = 313
> >> stat64("tmp/1.jar", {st_mode=S_IFREG|0644, st_size=2676, ...}) = 0
> >> open("tmp/1.jar", O_RDONLY)             = 49
> >> fcntl64(49, F_GETFD)                    = 0
> >> fcntl64(49, F_SETFD, FD_CLOEXEC)        = 0
> >> read(49, "PK\3\4", 4)                   = 4
> >> lseek(49, 0, SEEK_SET)                  = 0
> >> stat64("tmp/1.jar", {st_mode=S_IFREG|0644, st_size=2676, ...}) = 0
> >> stat64("tmp/1.jar", {st_mode=S_IFREG|0644, st_size=2676, ...}) = 0
> >> lseek(49, 0, SEEK_END)                  = 2676
> >> lseek(49, 1652, SEEK_SET)               = 1652
> >> read(49, "hH\304D|\254\340\223*g\2*\346u\3339\301\35>\5
> >> \265Z2-7v"..., 1024) = 1024
> >> lseek(49, 2345, SEEK_SET)               = 2345
> >> read(49, "PK\1\2\24\0\24\0\10\0\10\0\23\230\2325\0\0\0\0\2\0\0
> >> \0"..., 313) = 313
> >> stat64("tmp/2.jar", {st_mode=S_IFREG|0644, st_size=2676, ...}) = 0
> >> open("tmp/2.jar", O_RDONLY)             = 50
> >> fcntl64(50, F_GETFD)                    = 0
> >> fcntl64(50, F_SETFD, FD_CLOEXEC)        = 0
> >> read(50, "PK\3\4", 4)                   = 4
> >> lseek(50, 0, SEEK_SET)                  = 0
> >> stat64("tmp/2.jar", {st_mode=S_IFREG|0644, st_size=2676, ...}) = 0
> >> stat64("tmp/2.jar", {st_mode=S_IFREG|0644, st_size=2676, ...}) = 0
> >> lseek(50, 0, SEEK_END)                  = 2676
> >> lseek(50, 1652, SEEK_SET)               = 1652
> >> read(50, "hH\304D|\254\340\223*g\2*\346u\3339\301\35>\5
> >> \265Z2-7v"..., 1024) = 1024
> >> lseek(50, 2345, SEEK_SET)               = 2345
> >> read(50, "PK\1\2\24\0\24\0\10\0\10\0\23\230\2325\0\0\0\0\2\0\0
> >> \0"..., 313) = 313
> >> stat64("tmp/3.jar", {st_mode=S_IFREG|0644, st_size=2676, ...}) = 0
> >> open("tmp/3.jar", O_RDONLY)
> >> ---------------------------------
> >> - no "close" call.
> >>
> >>
> >> --
> >> This message is automatically generated by JIRA.
> >> -
> >> If you think it was sent incorrectly contact one of the
> >> administrators: http://issues.apache.org/jira/secure/
> >> Administrators.jspa
> >> -
> >> For more information on JIRA, see: http://www.atlassian.com/
> >> software/jira
> >>
> >>
> >>
> >
> >
> > --
> > Tony Wu
> > China Software Development Lab, IBM
>
>


-- 
Tony Wu
China Software Development Lab, IBM

Mime
View raw message