harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Geir Magnusson Jr." <g...@pobox.com>
Subject Re: [jira] Created: (HARMONY-2914) [classlib][archive] RFE: java.util.zip.ZipFile(File file) wastes file handlers on Linux
Date Wed, 24 Jan 2007 14:40:47 GMT

On Jan 24, 2007, at 3:36 AM, Tony Wu wrote:

> I think it is implementation detail. Harmony holds the handler when
> the ZipFile is created. For performance concern, it is ok to follow
> RI. What's your opinion?
>
> And I would argue that it is not very proper to use something like
> strace to detect the detail behavior of RI.

Why?

>
> On 12/28/06, Nina Rinskaya (JIRA) <jira@apache.org> wrote:
>> [classlib][archive] RFE: java.util.zip.ZipFile(File file) wastes  
>> file handlers on Linux
>> --------------------------------------------------------------------- 
>> ------------------
>>
>>                 Key: HARMONY-2914
>>                 URL: http://issues.apache.org/jira/browse/ 
>> HARMONY-2914
>>             Project: Harmony
>>          Issue Type: Wish
>>          Components: Classlib
>>         Environment: Linux (SLES9), ia32
>>            Reporter: Nina Rinskaya
>>            Priority: Minor
>>
>>
>> During Eclipse Unit Tests runs evaluation I've encountered an  
>> issue that looks like file handlers waste/leak on Linux when  
>> opening new zip (jar) file (using java.util.zip.Zipfile(File file)  
>> constructor): unlike RI (Sun), Harmony takes and doesn't releases  
>> file handlers when opening zip (jar) files. When working with  
>> large application with thousands of plugins and jar files in the  
>> classpath, the OS file handlers limit can be reached very quickly  
>> as it happens with some Eclipse Unit Tests on Harmony.
>>
>> The issue can be workarounded (and I did it in EUT) by calling GC  
>> each time we expect many zip/jar files opened.
>>
>> Note, JRockit 1.5 behaves almost in the same way as Harmony (goes  
>> out of file handlers and starts throwing IOException), but Sun  
>> works ok. I don't know how they did it, but it would be great if  
>> we make it work too.
>>
>> Here is the simple way to reproduce the issue:
>>
>> 1. Create 'tmp' subdirectory in the working directory, and copy  
>> any jar file 100 times under '$i.jar' names into 'tmp' subdirectory.
>> 2. Set low open files limit: ulimit -n 60 (reproducible with any  
>> limit actually)
>> 3. Compile and run the following test (FileTest.java):
>>
>> --------------------------------
>> import java.io.*;
>> import java.util.zip.ZipFile;
>>
>> public class FileTest {
>>    public static Object[] oo = new Object[100];
>>    public void testB() {
>>        for (int i=0; i<100; i++) {
>>            try {
>>                oo[i] = new ZipFile(new File("tmp/"+i+".jar"));
>>            } catch (IOException ioe) {
>>                ioe.printStackTrace();
>>                System.out.println("Failed on iteration " + i);
>>                return;
>>            }
>>        }
>>        System.out.println("OK");
>>    }
>>    public static void main(String[] args) {
>>        FileTest ft = new FileTest();
>>        ft.testB();
>>    }
>> }
>>
>> ---------------------------------
>>
>> Output on Harmony (looks like SIGABRT is caused by some VM actions  
>> during exit with open files limit reached - can also be a problem) :
>>
>> java.util.zip.ZipException: archive.24
>>        at java.util.zip.ZipFile.openZip(ZipFile.java:120)
>>        at java.util.zip.ZipFile.<init>(ZipFile.java:111)
>>        at java.util.zip.ZipFile.<init>(ZipFile.java:71)
>>        at FileTest2.testB(FileTest2.java:9)
>>        at FileTest2.main(FileTest2.java:18)
>> Failed on iteration 12
>> java: /export/users/nrinskay/harmony_build/trunk/working_vm/vm/ 
>> vmcore/src/util/linux/signals_ia32.cpp:254: void* find_stack_addr 
>> (): Assertion `!err' failed.
>> SIGABRT in VM code.
>> Stack trace:
>> Warning: Cannot get modules info, no symbolic information will be  
>> provided
>>        1: ?? (??:-1)
>> <end of stack trace>
>> Aborted
>> ----------------------------
>>
>> Output on JRockit 1.5:
>>
>> java.io.IOException: Error opening file
>>        at java.util.zip.ZipFile.open(Ljava.lang.String;I)I(Unknown  
>> Source)
>>        at java.util.zip.ZipFile.<init>(Ljava.io.File;I)V(Unknown  
>> Source)
>>        at java.util.zip.ZipFile.<init>(Ljava.io.File;)V(Unknown  
>> Source)
>>        at FileTest2.testB()V(FileTest2.java:9)
>>        at FileTest2.main([Ljava.lang.String;)V(FileTest2.java:20)
>> Failed on iteration 45
>> ------------------------------------
>>
>> Output on Sun 1.5:
>>
>> OK
>> ------------------------
>>
>>
>> I used strace to compare system calls logs for Sun 1.5, JRockit  
>> 1.5 and Harmony (see below). I'm not sure how it works, but Sun  
>> releases open files handles just after opening zip/jar files and  
>> never goes out of the limit.
>>
>> Sun:
>> -----------------------------
>> stat64("tmp/0.jar", {st_mode=S_IFREG|0644, st_size=2676, ...}) = 0
>> open("tmp/0.jar", O_RDONLY|O_LARGEFILE) = 3
>> fstat64(3, {st_mode=S_IFREG|0644, st_size=2676, ...}) = 0
>> _llseek(3, 0, [2676], SEEK_END)         = 0
>> mmap2(NULL, 2676, PROT_READ, MAP_SHARED, 3, 0) = 0x89e9a000
>> close(3)                                = 0
>> stat64("tmp/1.jar", {st_mode=S_IFREG|0644, st_size=2676, ...}) = 0
>> open("tmp/1.jar", O_RDONLY|O_LARGEFILE) = 3
>> fstat64(3, {st_mode=S_IFREG|0644, st_size=2676, ...}) = 0
>> _llseek(3, 0, [2676], SEEK_END)         = 0
>> mmap2(NULL, 2676, PROT_READ, MAP_SHARED, 3, 0) = 0x89e9b000
>> close(3)                                = 0
>> stat64("tmp/2.jar", {st_mode=S_IFREG|0644, st_size=2676, ...}) = 0
>> open("tmp/2.jar", O_RDONLY|O_LARGEFILE) = 3
>> fstat64(3, {st_mode=S_IFREG|0644, st_size=2676, ...}) = 0
>> _llseek(3, 0, [2676], SEEK_END)         = 0
>> mmap2(NULL, 2676, PROT_READ, MAP_SHARED, 3, 0) = 0x89e9c000
>> close(3)
>> -----------------------------
>>
>> We see "close" call for each opened file.
>>
>> Jrockit:
>> --------------------------
>> open("tmp/0.jar", O_RDONLY|O_LARGEFILE) = 15
>> fstat64(15, {st_mode=S_IFREG|0644, st_size=2676, ...}) = 0
>> pread(15, "PK\5\6\0\0\0\0\5\0\5\0005\1\0\0)\t\0\0\0\0", 22, 2654)  
>> = 22
>> pread(15, "PK\1\2\24\0\24\0\10\0\10\0\23\230\2325\0\0\0\0\2\0\0 
>> \0"..., 309, 2345) = 309
>> stat64("tmp/1.jar", {st_mode=S_IFREG|0644, st_size=2676, ...}) = 0
>> open("tmp/1.jar", O_RDONLY|O_LARGEFILE) = 16
>> fstat64(16, {st_mode=S_IFREG|0644, st_size=2676, ...}) = 0
>> pread(16, "PK\5\6\0\0\0\0\5\0\5\0005\1\0\0)\t\0\0\0\0", 22, 2654)  
>> = 22
>> pread(16, "PK\1\2\24\0\24\0\10\0\10\0\23\230\2325\0\0\0\0\2\0\0 
>> \0"..., 309, 2345) = 309
>> stat64("tmp/2.jar", {st_mode=S_IFREG|0644, st_size=2676, ...}) = 0
>> open("tmp/2.jar", O_RDONLY|O_LARGEFILE) = 17
>> fstat64(17, {st_mode=S_IFREG|0644, st_size=2676, ...}) = 0
>> pread(17, "PK\5\6\0\0\0\0\5\0\5\0005\1\0\0)\t\0\0\0\0", 22, 2654)  
>> = 22
>> pread(17, "PK\1\2\24\0\24\0\10\0\10\0\23\230\2325\0\0\0\0\2\0\0 
>> \0"..., 309, 2345) = 309
>> stat64("tmp/3.jar", {st_mode=S_IFREG|0644, st_size=2676, ...}) = 0
>> open("tmp/3.jar", O_RDONLY|O_LARGEFILE) = 18
>> ------------------------------
>>
>> - no "close" call.
>>
>> Harmony:
>> ------------------------------
>> open("tmp/0.jar", O_RDONLY)             = 48
>> fcntl64(48, F_GETFD)                    = 0
>> fcntl64(48, F_SETFD, FD_CLOEXEC)        = 0
>> read(48, "PK\3\4", 4)                   = 4
>> lseek(48, 0, SEEK_SET)                  = 0
>> stat64("tmp/0.jar", {st_mode=S_IFREG|0644, st_size=2676, ...}) = 0
>> stat64("tmp/0.jar", {st_mode=S_IFREG|0644, st_size=2676, ...}) = 0
>> lseek(48, 0, SEEK_END)                  = 2676
>> lseek(48, 1652, SEEK_SET)               = 1652
>> read(48, "hH\304D|\254\340\223*g\2*\346u\3339\301\35>\5 
>> \265Z2-7v"..., 1024) = 1024
>> lseek(48, 2345, SEEK_SET)               = 2345
>> read(48, "PK\1\2\24\0\24\0\10\0\10\0\23\230\2325\0\0\0\0\2\0\0 
>> \0"..., 313) = 313
>> stat64("tmp/1.jar", {st_mode=S_IFREG|0644, st_size=2676, ...}) = 0
>> open("tmp/1.jar", O_RDONLY)             = 49
>> fcntl64(49, F_GETFD)                    = 0
>> fcntl64(49, F_SETFD, FD_CLOEXEC)        = 0
>> read(49, "PK\3\4", 4)                   = 4
>> lseek(49, 0, SEEK_SET)                  = 0
>> stat64("tmp/1.jar", {st_mode=S_IFREG|0644, st_size=2676, ...}) = 0
>> stat64("tmp/1.jar", {st_mode=S_IFREG|0644, st_size=2676, ...}) = 0
>> lseek(49, 0, SEEK_END)                  = 2676
>> lseek(49, 1652, SEEK_SET)               = 1652
>> read(49, "hH\304D|\254\340\223*g\2*\346u\3339\301\35>\5 
>> \265Z2-7v"..., 1024) = 1024
>> lseek(49, 2345, SEEK_SET)               = 2345
>> read(49, "PK\1\2\24\0\24\0\10\0\10\0\23\230\2325\0\0\0\0\2\0\0 
>> \0"..., 313) = 313
>> stat64("tmp/2.jar", {st_mode=S_IFREG|0644, st_size=2676, ...}) = 0
>> open("tmp/2.jar", O_RDONLY)             = 50
>> fcntl64(50, F_GETFD)                    = 0
>> fcntl64(50, F_SETFD, FD_CLOEXEC)        = 0
>> read(50, "PK\3\4", 4)                   = 4
>> lseek(50, 0, SEEK_SET)                  = 0
>> stat64("tmp/2.jar", {st_mode=S_IFREG|0644, st_size=2676, ...}) = 0
>> stat64("tmp/2.jar", {st_mode=S_IFREG|0644, st_size=2676, ...}) = 0
>> lseek(50, 0, SEEK_END)                  = 2676
>> lseek(50, 1652, SEEK_SET)               = 1652
>> read(50, "hH\304D|\254\340\223*g\2*\346u\3339\301\35>\5 
>> \265Z2-7v"..., 1024) = 1024
>> lseek(50, 2345, SEEK_SET)               = 2345
>> read(50, "PK\1\2\24\0\24\0\10\0\10\0\23\230\2325\0\0\0\0\2\0\0 
>> \0"..., 313) = 313
>> stat64("tmp/3.jar", {st_mode=S_IFREG|0644, st_size=2676, ...}) = 0
>> open("tmp/3.jar", O_RDONLY)
>> ---------------------------------
>> - no "close" call.
>>
>>
>> --
>> This message is automatically generated by JIRA.
>> -
>> If you think it was sent incorrectly contact one of the  
>> administrators: http://issues.apache.org/jira/secure/ 
>> Administrators.jspa
>> -
>> For more information on JIRA, see: http://www.atlassian.com/ 
>> software/jira
>>
>>
>>
>
>
> -- 
> Tony Wu
> China Software Development Lab, IBM


Mime
View raw message