xmlgraphics-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthias Reischenbacher (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (XGC-93) Calling DeflaterOutputStream.write(byte) byte by byte causes blocked threads in multi session-environments
Date Thu, 29 Oct 2015 19:35:27 GMT

    [ https://issues.apache.org/jira/browse/XGC-93?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14981116#comment-14981116
] 

Matthias Reischenbacher commented on XGC-93:
--------------------------------------------

Added a patch based on Andre's suggestion. This indeed can have a big performance impact.
In a local test case with around 30 PNG images the fix reduced the processing time during
PDF output from approx. 6 seconds to 1 second (!).

> Calling DeflaterOutputStream.write(byte) byte by byte causes blocked threads in multi
session-environments
> ----------------------------------------------------------------------------------------------------------
>
>                 Key: XGC-93
>                 URL: https://issues.apache.org/jira/browse/XGC-93
>             Project: XMLGraphicsCommons
>          Issue Type: Improvement
>          Components: postscript
>    Affects Versions: trunk
>         Environment: WebSphere Application Server 7.0 with AIX 7.1 and JDK 1.6.0
> fop.jar, Implementation-Version: 1.1, Build-Id: 20121016-224756-ICT (glenn [Mac OS X
10.8.1 x86_64, Java 1.6.0_35-b10-428-11M3811, Target Java 1.5])
> xmlgraphics-commons-1.5.jar, Implementation-Version: 1.5, Build-Id: 20121016-215324-ICT
(glenn [Mac OS X 10.8.1 x86_64, Java 1.6.0_35-b10-428-11M3811])
>            Reporter: Andre Klemann
>              Labels: easyfix, performance
>             Fix For: trunk
>
>         Attachments: xgc-93.patch
>
>
> The method org.apache.xmlgraphics.ps.ImageEncodingHelper.optimizedWriteTo(OutputStream)
from xmlgraphics-commons-1.5.jar calls the method DeflaterOutputStream.write(byte) byte by
byte (see lines 242 - 244):
>     private boolean optimizedWriteTo(OutputStream out)
>             throws IOException {
>         if (this.firstTileDump) {
>             Raster raster = image.getTile(0, 0);
>             DataBuffer buffer = raster.getDataBuffer();
>             if (buffer instanceof DataBufferByte) {
>                 byte[] bytes = ((DataBufferByte) buffer).getData();
>                 // see determineEncodingColorModel() to see why we permute B and R here
>                 if (isBGR) {
>                     for (int i = 0; i < bytes.length; i += 3) {
>                         out.write(bytes[i + 2]);  // 242
>                         out.write(bytes[i + 1]);  // 243
>                         out.write(bytes[i]);        // 244
>                     }
>                 } else {
>                     out.write(bytes);
>                 }
>                 return true;
>             }
>         }
>         return false;
>     }
> Under WebSphere, in a multi-session environment, this leads to warnings about possibly
hanging threads:
> [07.03.13 08:34:22:673 MEZ] 0000000d ThreadMonitor W   WSVR0605W: Thread "WebContainer
: 1303" (00000547) has been active for 165445 milliseconds and may be hung.  There is/are
1 thread(s) in total in the server that may be hung.
> 	at java.util.zip.Deflater.deflateBytes(Native Method)
> 	at java.util.zip.Deflater.deflate(Deflater.java:306)
> 	at java.util.zip.DeflaterOutputStream.deflate(DeflaterOutputStream.java:153)
> 	at java.util.zip.DeflaterOutputStream.write(DeflaterOutputStream.java:112)
> 	at java.util.zip.DeflaterOutputStream.write(DeflaterOutputStream.java:89)
> 	at org.apache.xmlgraphics.ps.ImageEncodingHelper.optimizedWriteTo(ImageEncodingHelper.java:242)
> 	...
> [07.03.13 08:38:18:886 MEZ] 00000547 ThreadMonitor W   WSVR0606W: Thread "WebContainer
: 1303" (00000547) was previously reported to be hung but has completed.  It was active for
approximately 401662 milliseconds.  There is/are 0 thread(s) in total in the server that still
may be hung.
> [07.03.13 08:40:22:723 MEZ] 0000002d ThreadMonitor W   WSVR0605W: Thread "WebContainer
: 1323" (0000055b) has been active for 221920 milliseconds and may be hung.  There is/are
1 thread(s) in total in the server that may be hung.
> 	at java.util.zip.Deflater.deflateBytes(Native Method)
> 	at java.util.zip.Deflater.deflate(Deflater.java:306)
> 	at java.util.zip.DeflaterOutputStream.deflate(DeflaterOutputStream.java:153)
> 	at java.util.zip.DeflaterOutputStream.write(DeflaterOutputStream.java:112)
> 	at java.util.zip.DeflaterOutputStream.write(DeflaterOutputStream.java:89)
> 	at org.apache.xmlgraphics.ps.ImageEncodingHelper.optimizedWriteTo(ImageEncodingHelper.java:244)
> 	...
> The threads will finish their work, but this takes a long time.
> The reason for this is, that the number n of calls to java.util.zip.DeflaterOutputStream.write(byte)
might be in a hight range. I have seen cases, where n=4094496. This leads to n calls of DeflaterOutputSteam.write(byte[]
b, int off, int len), where off=0 and len=1:
>     public void write(byte[] b, int off, int len) throws IOException {
> 	if (def.finished()) {
> 	    throw new IOException("write beyond end of stream");
> 	}
>         if ((off | len | (off + len) | (b.length - (off + len))) < 0) {
> 	    throw new IndexOutOfBoundsException();
> 	} else if (len == 0) {
> 	    return;
> 	}
> 	if (!def.finished()) {
>             // Deflate no more than stride bytes at a time.  This avoids
>             // excess copying in deflateBytes (see Deflater.c)
>             int stride = buf.length;
>             for (int i = 0; i < len; i+= stride) {
>                 def.setInput(b, off + i, Math.min(stride, len - i));   // line 116
>                 while (!def.needsInput()) {
>                     deflate();                                                      
  // line 118
>                 }
>             }
> 	}
>     }
> Since java.util.zip.Deflater internally uses synchronized blocks, esspecially in setInput(byte[],
int, int) and deflate(byte[], int, int), one should reduce the number of calls to these methods
as much as possible.
> The number of calls of Deflater.setInput(byte[] b, int off, int len) in line 116 
> above could, e.g., be reduced by the factor 512 (=buffer.length), if the method ImageEncodingHelper.optimizedWriteTo()
would be written in this way:
>     private boolean optimizedWriteTo(OutputStream out)
>             throws IOException {
>         if (this.firstTileDump) {
>             Raster raster = image.getTile(0, 0);
>             DataBuffer buffer = raster.getDataBuffer();
>             if (buffer instanceof DataBufferByte) {
>                 byte[] bytes = ((DataBufferByte) buffer).getData();
>                 // see determineEncodingColorModel() to see why we permute B and R here
>                 if (isBGR) {
>                     byte[] bytesPermutated = new byte[bytes.length];
>                     for (int i = 0; i < bytes.length; i += 3) {
>                         bytesPermutated[i] = bytes[i+2];
>                         bytesPermutated[i+1] = bytes[i+1];
>                         bytesPermutated[i+2] = bytes[i];
>                     }
>                     out.write(bytesPermutated);
>                 } else {
>                     out.write(bytes);
>                 }
>                 return true;
>             }
>         }
>         return false;
>     }
> In this version, n calls of DeflaterOutputStream.write(byte) are replaced by one call
of DeflaterOutputStream(byte[]), which is much faster and offers less blocking potential.
> Without this optimization and with activated zlib compression in FOP 1.1, we always got
warnings from WeSphere's thread monitor, that some threads may hang.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: general-help@xmlgraphics.apache.org


Mime
View raw message