harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Geir Magnusson Jr." <g...@pobox.com>
Subject Re: [jira] Created: (HARMONY-2914) [classlib][archive] RFE: java.util.zip.ZipFile(File file) wastes file handlers on Linux
Date Wed, 24 Jan 2007 15:42:37 GMT
but strace simply tells you about system behavior at runtime right?   
no source is involved?

geir

On Jan 24, 2007, at 10:33 AM, Tony Wu wrote:

> IIRC, the ACQ we have signed made some restrictions on touching the
> source code of RI. I thought the activities which may be considered as
> "accessing the source code" should be prevented, otherwise one would
> disqualify from committing to the repository here at Apache Harmony.
>
> If you think that it is ok to do that, it would be ok for me also.
>
> On 1/24/07, Geir Magnusson Jr. <geir@pobox.com> wrote:
>>
>> On Jan 24, 2007, at 3:36 AM, Tony Wu wrote:
>>
>> > I think it is implementation detail. Harmony holds the handler when
>> > the ZipFile is created. For performance concern, it is ok to follow
>> > RI. What's your opinion?
>> >
>> > And I would argue that it is not very proper to use something like
>> > strace to detect the detail behavior of RI.
>>
>> Why?
>>
>> >
>> > On 12/28/06, Nina Rinskaya (JIRA) <jira@apache.org> wrote:
>> >> [classlib][archive] RFE: java.util.zip.ZipFile(File file) wastes
>> >> file handlers on Linux
>> >>  
>> ---------------------------------------------------------------------
>> >> ------------------
>> >>
>> >>                 Key: HARMONY-2914
>> >>                 URL: http://issues.apache.org/jira/browse/
>> >> HARMONY-2914
>> >>             Project: Harmony
>> >>          Issue Type: Wish
>> >>          Components: Classlib
>> >>         Environment: Linux (SLES9), ia32
>> >>            Reporter: Nina Rinskaya
>> >>            Priority: Minor
>> >>
>> >>
>> >> During Eclipse Unit Tests runs evaluation I've encountered an
>> >> issue that looks like file handlers waste/leak on Linux when
>> >> opening new zip (jar) file (using java.util.zip.Zipfile(File file)
>> >> constructor): unlike RI (Sun), Harmony takes and doesn't releases
>> >> file handlers when opening zip (jar) files. When working with
>> >> large application with thousands of plugins and jar files in the
>> >> classpath, the OS file handlers limit can be reached very quickly
>> >> as it happens with some Eclipse Unit Tests on Harmony.
>> >>
>> >> The issue can be workarounded (and I did it in EUT) by calling GC
>> >> each time we expect many zip/jar files opened.
>> >>
>> >> Note, JRockit 1.5 behaves almost in the same way as Harmony (goes
>> >> out of file handlers and starts throwing IOException), but Sun
>> >> works ok. I don't know how they did it, but it would be great if
>> >> we make it work too.
>> >>
>> >> Here is the simple way to reproduce the issue:
>> >>
>> >> 1. Create 'tmp' subdirectory in the working directory, and copy
>> >> any jar file 100 times under '$i.jar' names into 'tmp'  
>> subdirectory.
>> >> 2. Set low open files limit: ulimit -n 60 (reproducible with any
>> >> limit actually)
>> >> 3. Compile and run the following test (FileTest.java):
>> >>
>> >> --------------------------------
>> >> import java.io.*;
>> >> import java.util.zip.ZipFile;
>> >>
>> >> public class FileTest {
>> >>    public static Object[] oo = new Object[100];
>> >>    public void testB() {
>> >>        for (int i=0; i<100; i++) {
>> >>            try {
>> >>                oo[i] = new ZipFile(new File("tmp/"+i+".jar"));
>> >>            } catch (IOException ioe) {
>> >>                ioe.printStackTrace();
>> >>                System.out.println("Failed on iteration " + i);
>> >>                return;
>> >>            }
>> >>        }
>> >>        System.out.println("OK");
>> >>    }
>> >>    public static void main(String[] args) {
>> >>        FileTest ft = new FileTest();
>> >>        ft.testB();
>> >>    }
>> >> }
>> >>
>> >> ---------------------------------
>> >>
>> >> Output on Harmony (looks like SIGABRT is caused by some VM actions
>> >> during exit with open files limit reached - can also be a  
>> problem) :
>> >>
>> >> java.util.zip.ZipException: archive.24
>> >>        at java.util.zip.ZipFile.openZip(ZipFile.java:120)
>> >>        at java.util.zip.ZipFile.<init>(ZipFile.java:111)
>> >>        at java.util.zip.ZipFile.<init>(ZipFile.java:71)
>> >>        at FileTest2.testB(FileTest2.java:9)
>> >>        at FileTest2.main(FileTest2.java:18)
>> >> Failed on iteration 12
>> >> java: /export/users/nrinskay/harmony_build/trunk/working_vm/vm/
>> >> vmcore/src/util/linux/signals_ia32.cpp:254: void* find_stack_addr
>> >> (): Assertion `!err' failed.
>> >> SIGABRT in VM code.
>> >> Stack trace:
>> >> Warning: Cannot get modules info, no symbolic information will be
>> >> provided
>> >>        1: ?? (??:-1)
>> >> <end of stack trace>
>> >> Aborted
>> >> ----------------------------
>> >>
>> >> Output on JRockit 1.5:
>> >>
>> >> java.io.IOException: Error opening file
>> >>        at java.util.zip.ZipFile.open(Ljava.lang.String;I)I(Unknown
>> >> Source)
>> >>        at java.util.zip.ZipFile.<init>(Ljava.io.File;I)V(Unknown
>> >> Source)
>> >>        at java.util.zip.ZipFile.<init>(Ljava.io.File;)V(Unknown
>> >> Source)
>> >>        at FileTest2.testB()V(FileTest2.java:9)
>> >>        at FileTest2.main([Ljava.lang.String;)V(FileTest2.java:20)
>> >> Failed on iteration 45
>> >> ------------------------------------
>> >>
>> >> Output on Sun 1.5:
>> >>
>> >> OK
>> >> ------------------------
>> >>
>> >>
>> >> I used strace to compare system calls logs for Sun 1.5, JRockit
>> >> 1.5 and Harmony (see below). I'm not sure how it works, but Sun
>> >> releases open files handles just after opening zip/jar files and
>> >> never goes out of the limit.
>> >>
>> >> Sun:
>> >> -----------------------------
>> >> stat64("tmp/0.jar", {st_mode=S_IFREG|0644, st_size=2676, ...}) = 0
>> >> open("tmp/0.jar", O_RDONLY|O_LARGEFILE) = 3
>> >> fstat64(3, {st_mode=S_IFREG|0644, st_size=2676, ...}) = 0
>> >> _llseek(3, 0, [2676], SEEK_END)         = 0
>> >> mmap2(NULL, 2676, PROT_READ, MAP_SHARED, 3, 0) = 0x89e9a000
>> >> close(3)                                = 0
>> >> stat64("tmp/1.jar", {st_mode=S_IFREG|0644, st_size=2676, ...}) = 0
>> >> open("tmp/1.jar", O_RDONLY|O_LARGEFILE) = 3
>> >> fstat64(3, {st_mode=S_IFREG|0644, st_size=2676, ...}) = 0
>> >> _llseek(3, 0, [2676], SEEK_END)         = 0
>> >> mmap2(NULL, 2676, PROT_READ, MAP_SHARED, 3, 0) = 0x89e9b000
>> >> close(3)                                = 0
>> >> stat64("tmp/2.jar", {st_mode=S_IFREG|0644, st_size=2676, ...}) = 0
>> >> open("tmp/2.jar", O_RDONLY|O_LARGEFILE) = 3
>> >> fstat64(3, {st_mode=S_IFREG|0644, st_size=2676, ...}) = 0
>> >> _llseek(3, 0, [2676], SEEK_END)         = 0
>> >> mmap2(NULL, 2676, PROT_READ, MAP_SHARED, 3, 0) = 0x89e9c000
>> >> close(3)
>> >> -----------------------------
>> >>
>> >> We see "close" call for each opened file.
>> >>
>> >> Jrockit:
>> >> --------------------------
>> >> open("tmp/0.jar", O_RDONLY|O_LARGEFILE) = 15
>> >> fstat64(15, {st_mode=S_IFREG|0644, st_size=2676, ...}) = 0
>> >> pread(15, "PK\5\6\0\0\0\0\5\0\5\0005\1\0\0)\t\0\0\0\0", 22, 2654)
>> >> = 22
>> >> pread(15, "PK\1\2\24\0\24\0\10\0\10\0\23\230\2325\0\0\0\0\2\0\0
>> >> \0"..., 309, 2345) = 309
>> >> stat64("tmp/1.jar", {st_mode=S_IFREG|0644, st_size=2676, ...}) = 0
>> >> open("tmp/1.jar", O_RDONLY|O_LARGEFILE) = 16
>> >> fstat64(16, {st_mode=S_IFREG|0644, st_size=2676, ...}) = 0
>> >> pread(16, "PK\5\6\0\0\0\0\5\0\5\0005\1\0\0)\t\0\0\0\0", 22, 2654)
>> >> = 22
>> >> pread(16, "PK\1\2\24\0\24\0\10\0\10\0\23\230\2325\0\0\0\0\2\0\0
>> >> \0"..., 309, 2345) = 309
>> >> stat64("tmp/2.jar", {st_mode=S_IFREG|0644, st_size=2676, ...}) = 0
>> >> open("tmp/2.jar", O_RDONLY|O_LARGEFILE) = 17
>> >> fstat64(17, {st_mode=S_IFREG|0644, st_size=2676, ...}) = 0
>> >> pread(17, "PK\5\6\0\0\0\0\5\0\5\0005\1\0\0)\t\0\0\0\0", 22, 2654)
>> >> = 22
>> >> pread(17, "PK\1\2\24\0\24\0\10\0\10\0\23\230\2325\0\0\0\0\2\0\0
>> >> \0"..., 309, 2345) = 309
>> >> stat64("tmp/3.jar", {st_mode=S_IFREG|0644, st_size=2676, ...}) = 0
>> >> open("tmp/3.jar", O_RDONLY|O_LARGEFILE) = 18
>> >> ------------------------------
>> >>
>> >> - no "close" call.
>> >>
>> >> Harmony:
>> >> ------------------------------
>> >> open("tmp/0.jar", O_RDONLY)             = 48
>> >> fcntl64(48, F_GETFD)                    = 0
>> >> fcntl64(48, F_SETFD, FD_CLOEXEC)        = 0
>> >> read(48, "PK\3\4", 4)                   = 4
>> >> lseek(48, 0, SEEK_SET)                  = 0
>> >> stat64("tmp/0.jar", {st_mode=S_IFREG|0644, st_size=2676, ...}) = 0
>> >> stat64("tmp/0.jar", {st_mode=S_IFREG|0644, st_size=2676, ...}) = 0
>> >> lseek(48, 0, SEEK_END)                  = 2676
>> >> lseek(48, 1652, SEEK_SET)               = 1652
>> >> read(48, "hH\304D|\254\340\223*g\2*\346u\3339\301\35>\5
>> >> \265Z2-7v"..., 1024) = 1024
>> >> lseek(48, 2345, SEEK_SET)               = 2345
>> >> read(48, "PK\1\2\24\0\24\0\10\0\10\0\23\230\2325\0\0\0\0\2\0\0
>> >> \0"..., 313) = 313
>> >> stat64("tmp/1.jar", {st_mode=S_IFREG|0644, st_size=2676, ...}) = 0
>> >> open("tmp/1.jar", O_RDONLY)             = 49
>> >> fcntl64(49, F_GETFD)                    = 0
>> >> fcntl64(49, F_SETFD, FD_CLOEXEC)        = 0
>> >> read(49, "PK\3\4", 4)                   = 4
>> >> lseek(49, 0, SEEK_SET)                  = 0
>> >> stat64("tmp/1.jar", {st_mode=S_IFREG|0644, st_size=2676, ...}) = 0
>> >> stat64("tmp/1.jar", {st_mode=S_IFREG|0644, st_size=2676, ...}) = 0
>> >> lseek(49, 0, SEEK_END)                  = 2676
>> >> lseek(49, 1652, SEEK_SET)               = 1652
>> >> read(49, "hH\304D|\254\340\223*g\2*\346u\3339\301\35>\5
>> >> \265Z2-7v"..., 1024) = 1024
>> >> lseek(49, 2345, SEEK_SET)               = 2345
>> >> read(49, "PK\1\2\24\0\24\0\10\0\10\0\23\230\2325\0\0\0\0\2\0\0
>> >> \0"..., 313) = 313
>> >> stat64("tmp/2.jar", {st_mode=S_IFREG|0644, st_size=2676, ...}) = 0
>> >> open("tmp/2.jar", O_RDONLY)             = 50
>> >> fcntl64(50, F_GETFD)                    = 0
>> >> fcntl64(50, F_SETFD, FD_CLOEXEC)        = 0
>> >> read(50, "PK\3\4", 4)                   = 4
>> >> lseek(50, 0, SEEK_SET)                  = 0
>> >> stat64("tmp/2.jar", {st_mode=S_IFREG|0644, st_size=2676, ...}) = 0
>> >> stat64("tmp/2.jar", {st_mode=S_IFREG|0644, st_size=2676, ...}) = 0
>> >> lseek(50, 0, SEEK_END)                  = 2676
>> >> lseek(50, 1652, SEEK_SET)               = 1652
>> >> read(50, "hH\304D|\254\340\223*g\2*\346u\3339\301\35>\5
>> >> \265Z2-7v"..., 1024) = 1024
>> >> lseek(50, 2345, SEEK_SET)               = 2345
>> >> read(50, "PK\1\2\24\0\24\0\10\0\10\0\23\230\2325\0\0\0\0\2\0\0
>> >> \0"..., 313) = 313
>> >> stat64("tmp/3.jar", {st_mode=S_IFREG|0644, st_size=2676, ...}) = 0
>> >> open("tmp/3.jar", O_RDONLY)
>> >> ---------------------------------
>> >> - no "close" call.
>> >>
>> >>
>> >> --
>> >> This message is automatically generated by JIRA.
>> >> -
>> >> If you think it was sent incorrectly contact one of the
>> >> administrators: http://issues.apache.org/jira/secure/
>> >> Administrators.jspa
>> >> -
>> >> For more information on JIRA, see: http://www.atlassian.com/
>> >> software/jira
>> >>
>> >>
>> >>
>> >
>> >
>> > --
>> > Tony Wu
>> > China Software Development Lab, IBM
>>
>>
>
>
> -- 
> Tony Wu
> China Software Development Lab, IBM


Mime
View raw message