Mailing-List: contact dev-help@harmony.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@harmony.apache.org
Received-SPF: pass (nike.apache.org: domain of enh@google.com designates
 216.239.45.13 as permitted sender)
DomainKey-Signature: a=rsa-sha1; s=beta; d=google.com; c=nofws; q=dns;
	h=mime-version:in-reply-to:references:date:message-id:subject:from:to:
	content-type:content-transfer-encoding:x-system-of-record;
	b=bzK0tbBl6SmawDVDvv1lax5bIMxK2WzDVd/3+V7cRvZjDGRJ/nTEKS8cD0EnoQ0UJ
	wGkSs9ZNI7DtU1tXhkxsA==
MIME-Version: 1.0
In-Reply-To: <4B0EF08B.2050605@gmail.com>
References: <4B0BEE4F.1080601@gmail.com> <4B0C05A2.5020501@googlemail.com>
	 <4B0E6F23.8070009@googlemail.com>
	 <200911261245.nAQCjwbj010629@d06av01.portsmouth.uk.ibm.com>
	 <200911261314.nAQDEYXf004654@d06av04.portsmouth.uk.ibm.com>
	 <4B0E8E27.50604@gmail.com> <4B0E9832.2040608@googlemail.com>
	 <4B0EB41E.7030201@googlemail.com> <4B0ECCAB.6000706@gmail.com>
	 <4B0EF08B.2050605@gmail.com>
Date: Thu, 26 Nov 2009 21:29:06 -0800
Message-ID: <96933a4d0911262129v37ed3391t782ac4d09995e1f1@mail.gmail.com>
Subject: Re: [classlib][archive] problems with ZipFile 9was: Re: [testing] M12
	testing on Windows x86)
From: enh <enh@google.com>
To: dev@harmony.apache.org
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

may as well fix the lot, rather than sniff these out one by one...

./rmi/src/main/java/java/rmi/server/RMIClassLoader.java:214:
byte[] buf =3D new byte[in.available()];
./luni/src/main/java/org/apache/harmony/luni/util/InputStreamExposer.java:8=
6:
               buffer =3D new byte[available];
./luni/src/main/java/org/apache/harmony/luni/util/ExposedByteArrayInputStre=
am.java:52:
       final byte[] buffer =3D new byte[available];
./luni/src/test/api/common/org/apache/harmony/luni/tests/java/io/Serializat=
ionStressTest.java:670:
              byte[] input =3D new byte[dis.available()];
./print/src/test/api/java/common/javax/print/ValueTests.java:562:
  res =3D new byte [iStream.available()];
./security/src/main/java/common/org/apache/harmony/security/utils/JarUtils.=
java:147:
       byte[] sfBytes =3D new byte[signature.available()];
./swing/src/main/java/common/javax/swing/plaf/basic/BasicLookAndFeel.java:1=
022:
                       byte[] data =3D new
byte[audioStream.available()];
./swing/src/main/java/common/javax/swing/text/html/parser/DTD.java:145:
       byte[] enc =3D new byte[stream.available()];

 --elliott

On Thu, Nov 26, 2009 at 13:18, Tim Ellison <t.p.ellison@gmail.com> wrote:
> On 26/Nov/2009 18:44, Tim Ellison wrote:
>> On 26/Nov/2009 17:00, Oliver Deakin wrote:
>>> A little more progress.
>>> It looks like we're reading javax\swing\text\html\parser\html32.bdtd
>>> (which is where the ZipFile/ZipEntry classes come in). I see that with
>>> the M12 version of those classes we end up throwing an ASN1Exception at
>>> line 886 of BerInputStream.java (right at the start of it's read()
>>> method). This unravels the stack up to ParserDelegator.createDTD() whic=
h
>>> has a "try ... catch(Exception e) { // do nothing }" around the DTD
>>> read. The exception is ignored and we carry on, but the DTD has not bee=
n
>>> fully populated and it looks like this is the cause of the html not
>>> being processed correctly and the tests failing. Needless to say, this
>>> exception is not thrown with the M11 version of ZipFile/ZipEntry.
>>>
>>> The question now is why offset=3D=3Dbuffer.length at the start of the
>>> BerInputStream.read() call with the new ZipFile/ZipEntry classes. Ill
>>> keep digging.
>>
>> I'll bet you a beer that it is our friend available() again...
>>
>> Consider this:
>>
>> =C2=A0static final String NAME =3D "javax/swing/text/html/parser/html32.=
bdtd";
>>
>> =C2=A0public void test() throws IOException {
>> =C2=A0 =C2=A0 JarFile jar =3D new JarFile("swing.jar");
>> =C2=A0 =C2=A0 ZipEntry ze =3D jar.getEntry(NAME);
>> =C2=A0 =C2=A0 InputStream is =3D jar.getInputStream(ze);
>>
>> =C2=A0 =C2=A0 System.out.println("Size =3D " + ze.getSize());
>> =C2=A0 =C2=A0 System.out.println("Available =3D " + is.available());
>>
>> =C2=A0 =C2=A0 jar.close();
>> =C2=A0}
>>
>> On Harmony it prints:
>> Size =3D 51140
>> Available =3D 1
>>
>> On the RI it prints:
>> Size =3D 51140
>> Available =3D 51140
>>
>> I know the arguments that say available() should not be used to judge
>> the total number of bytes that can be read, but it seems that a number
>> of applications (including our generated parser?) use it in this way.
>>
>> I haven't tried changing it yet[1], but I'll take that bet if you want :=
-)
>>
>> Regards,
>> Tim
>>
>> [1] I'm supposed to be packing for a long weekend away, so I'll be quiet
>> for the next few days.
>
> Ok, so I may owe you a beer :-)
>
> A quick trial of implementing available() for the inflater stream fixes
> this test case (above) but not the original problem.
>
> Given there are these differences we see emerging, I wonder if we should
> revert the offending code and retry again after M12?
>
> FYI here is my hacked available() impl, which needs some testing before
> it is good to go in..
>
> Index: src/main/java/java/util/zip/ZipFile.java
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
> --- src/main/java/java/util/zip/ZipFile.java =C2=A0 =C2=A0(revision 88454=
3)
> +++ src/main/java/java/util/zip/ZipFile.java =C2=A0 =C2=A0(working copy)
> @@ -247,7 +247,7 @@
> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 rafstrm.skip(entry.nameLen + lo=
calExtraLenOrWhatever);
> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 rafstrm.mLength =3D rafstrm.mOf=
fset + entry.compressedSize;
> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 if (entry.compressionMethod =3D=
=3D ZipEntry.DEFLATED) {
> - =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return new Infla=
terInputStream(rafstrm, new
> Inflater(true));
> + =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return new ZipIn=
flaterInputStream(rafstrm, new
> Inflater(true), 1024, entry);
> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 } else {
> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 return rafstrm;
> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 }
> @@ -415,4 +415,27 @@
> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 return n;
> =C2=A0 =C2=A0 =C2=A0 =C2=A0 }
> =C2=A0 =C2=A0 }
> +
> + =C2=A0 =C2=A0static class ZipInflaterInputStream extends InflaterInputS=
tream {
> +
> + =C2=A0 =C2=A0 =C2=A0 =C2=A0ZipEntry entry;
> + =C2=A0 =C2=A0 =C2=A0 =C2=A0long bytesRead =3D 0;
> +
> + =C2=A0 =C2=A0 =C2=A0 =C2=A0public ZipInflaterInputStream(InputStream is=
, Inflater inf, int
> bsize, ZipEntry entry) {
> + =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0super(is, inf, bsize);
> + =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0this.entry =3D entry;
> + =C2=A0 =C2=A0 =C2=A0 =C2=A0}
> +
> + =C2=A0 =C2=A0 =C2=A0 =C2=A0public int read(byte[] buffer, int off, int =
nbytes) throws
> IOException {
> + =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0int i =3D super.read(buffer, o=
ff, nbytes);
> + =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if (i !=3D -1) {
> + =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0bytesRead +=3D i=
;
> + =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0}
> + =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return i;
> + =C2=A0 =C2=A0 =C2=A0 =C2=A0}
> +
> + =C2=A0 =C2=A0 =C2=A0 =C2=A0public int available() throws IOException {
> + =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return super.available() =3D=
=3D 0 ? 0 : (int) (entry.getSize()
> - bytesRead);
> + =C2=A0 =C2=A0 =C2=A0 =C2=A0}
> + =C2=A0 =C2=A0}
> =C2=A0}
>
>
>


--=20
Elliott Hughes - http://who/enh - http://jessies.org/~enh/