xmlgraphics-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andre Klemann (JIRA)" <j...@apache.org>
Subject [jira] [Created] (XGC-93) Calling DeflaterOutputStream.write(byte) byte by byte causes blocked threads in multi session-environments
Date Wed, 17 Sep 2014 13:31:33 GMT
Andre Klemann created XGC-93:
--------------------------------

             Summary: Calling DeflaterOutputStream.write(byte) byte by byte causes blocked
threads in multi session-environments
                 Key: XGC-93
                 URL: https://issues.apache.org/jira/browse/XGC-93
             Project: XMLGraphicsCommons
          Issue Type: Improvement
          Components: postscript
    Affects Versions: Trunk
         Environment: WebSphere Application Server 7.0 with AIX 7.1 and JDK 1.6.0
fop.jar, Implementation-Version: 1.1, Build-Id: 20121016-224756-ICT (glenn [Mac OS X 10.8.1
x86_64, Java 1.6.0_35-b10-428-11M3811, Target Java 1.5])
xmlgraphics-commons-1.5.jar, Implementation-Version: 1.5, Build-Id: 20121016-215324-ICT (glenn
[Mac OS X 10.8.1 x86_64, Java 1.6.0_35-b10-428-11M3811])
            Reporter: Andre Klemann
             Fix For: Trunk


The method org.apache.xmlgraphics.ps.ImageEncodingHelper.optimizedWriteTo(OutputStream) from
xmlgraphics-commons-1.5.jar calls the method DeflaterOutputStream.write(byte) byte by byte
(see lines 242 - 244):

    private boolean optimizedWriteTo(OutputStream out)
            throws IOException {
        if (this.firstTileDump) {
            Raster raster = image.getTile(0, 0);
            DataBuffer buffer = raster.getDataBuffer();
            if (buffer instanceof DataBufferByte) {
                byte[] bytes = ((DataBufferByte) buffer).getData();
                // see determineEncodingColorModel() to see why we permute B and R here
                if (isBGR) {
                    for (int i = 0; i < bytes.length; i += 3) {
                        out.write(bytes[i + 2]);  // 242
                        out.write(bytes[i + 1]);  // 243
                        out.write(bytes[i]);        // 244
                    }
                } else {
                    out.write(bytes);
                }
                return true;
            }
        }
        return false;
    }

Under WebSphere, in a multi-session environment, this leads to warnings about possibly hanging
threads:

[07.03.13 08:34:22:673 MEZ] 0000000d ThreadMonitor W   WSVR0605W: Thread "WebContainer : 1303"
(00000547) has been active for 165445 milliseconds and may be hung.  There is/are 1 thread(s)
in total in the server that may be hung.
	at java.util.zip.Deflater.deflateBytes(Native Method)
	at java.util.zip.Deflater.deflate(Deflater.java:306)
	at java.util.zip.DeflaterOutputStream.deflate(DeflaterOutputStream.java:153)
	at java.util.zip.DeflaterOutputStream.write(DeflaterOutputStream.java:112)
	at java.util.zip.DeflaterOutputStream.write(DeflaterOutputStream.java:89)
	at org.apache.xmlgraphics.ps.ImageEncodingHelper.optimizedWriteTo(ImageEncodingHelper.java:242)
	...
[07.03.13 08:38:18:886 MEZ] 00000547 ThreadMonitor W   WSVR0606W: Thread "WebContainer : 1303"
(00000547) was previously reported to be hung but has completed.  It was active for approximately
401662 milliseconds.  There is/are 0 thread(s) in total in the server that still may be hung.
[07.03.13 08:40:22:723 MEZ] 0000002d ThreadMonitor W   WSVR0605W: Thread "WebContainer : 1323"
(0000055b) has been active for 221920 milliseconds and may be hung.  There is/are 1 thread(s)
in total in the server that may be hung.
	at java.util.zip.Deflater.deflateBytes(Native Method)
	at java.util.zip.Deflater.deflate(Deflater.java:306)
	at java.util.zip.DeflaterOutputStream.deflate(DeflaterOutputStream.java:153)
	at java.util.zip.DeflaterOutputStream.write(DeflaterOutputStream.java:112)
	at java.util.zip.DeflaterOutputStream.write(DeflaterOutputStream.java:89)
	at org.apache.xmlgraphics.ps.ImageEncodingHelper.optimizedWriteTo(ImageEncodingHelper.java:244)
	...

The threads will finish their work, but this takes a long time.

The reason for this is, that the number n of calls to java.util.zip.DeflaterOutputStream.write(byte)
might be in a hight range. I have seen cases, where n=4094496. This leads to n calls of DeflaterOutputSteam.write(byte[]
b, int off, int len), where off=0 and len=1:

    public void write(byte[] b, int off, int len) throws IOException {
	if (def.finished()) {
	    throw new IOException("write beyond end of stream");
	}
        if ((off | len | (off + len) | (b.length - (off + len))) < 0) {
	    throw new IndexOutOfBoundsException();
	} else if (len == 0) {
	    return;
	}
	if (!def.finished()) {
            // Deflate no more than stride bytes at a time.  This avoids
            // excess copying in deflateBytes (see Deflater.c)
            int stride = buf.length;
            for (int i = 0; i < len; i+= stride) {
                def.setInput(b, off + i, Math.min(stride, len - i));   // line 116
                while (!def.needsInput()) {
                    deflate();                                                         //
line 118
                }
            }
	}
    }

Since java.util.zip.Deflater internally uses synchronized blocks, esspecially in setInput(byte[],
int, int) and deflate(byte[], int, int), one should reduce the number of calls to these methods
as much as possible.

The number of calls of Deflater.setInput(byte[] b, int off, int len) in line 116 
above could, e.g., be reduced by the factor 512 (=buffer.length), if the method ImageEncodingHelper.optimizedWriteTo()
would be written in this way:

    private boolean optimizedWriteTo(OutputStream out)
            throws IOException {
        if (this.firstTileDump) {
            Raster raster = image.getTile(0, 0);
            DataBuffer buffer = raster.getDataBuffer();
            if (buffer instanceof DataBufferByte) {
                byte[] bytes = ((DataBufferByte) buffer).getData();
                // see determineEncodingColorModel() to see why we permute B and R here
                if (isBGR) {
                    byte[] bytesPermutated = new byte[bytes.length];
                    for (int i = 0; i < bytes.length; i += 3) {
                        bytesPermutated[i] = bytes[i+2];
                        bytesPermutated[i+1] = bytes[i+1];
                        bytesPermutated[i+2] = bytes[i];
                    }
                    out.write(bytesPermutated);
                } else {
                    out.write(bytes);
                }
                return true;
            }
        }
        return false;
    }

In this version, n calls of DeflaterOutputStream.write(byte) are replaced by one call of DeflaterOutputStream(byte[]),
which is much faster and offers less blocking potential.

Without this optimization and with activated zlib compression in FOP 1.1, we always got warnings
from WeSphere's thread monitor, that some threads may hang.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: general-help@xmlgraphics.apache.org


Mime
View raw message