Return-Path: X-Original-To: apmail-xmlgraphics-general-archive@www.apache.org Delivered-To: apmail-xmlgraphics-general-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 5538418171 for ; Fri, 30 Oct 2015 13:53:34 +0000 (UTC) Received: (qmail 79911 invoked by uid 500); 30 Oct 2015 13:53:28 -0000 Mailing-List: contact general-help@xmlgraphics.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: general@xmlgraphics.apache.org Delivered-To: mailing list general@xmlgraphics.apache.org Received: (qmail 79878 invoked by uid 99); 30 Oct 2015 13:53:27 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 30 Oct 2015 13:53:27 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id CCE172C1F57 for ; Fri, 30 Oct 2015 13:53:27 +0000 (UTC) Date: Fri, 30 Oct 2015 13:53:27 +0000 (UTC) From: "Chris Bowditch (JIRA)" To: general@xmlgraphics.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (XGC-93) Calling DeflaterOutputStream.write(byte) byte by byte causes blocked threads in multi session-environments MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/XGC-93?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14982580#comment-14982580 ] Chris Bowditch commented on XGC-93: ----------------------------------- Hi [~matthias8283], please do assume lazy consensus and commit your patch if there is no feedback after a few days. > Calling DeflaterOutputStream.write(byte) byte by byte causes blocked threads in multi session-environments > ---------------------------------------------------------------------------------------------------------- > > Key: XGC-93 > URL: https://issues.apache.org/jira/browse/XGC-93 > Project: XMLGraphicsCommons > Issue Type: Improvement > Components: postscript > Affects Versions: trunk > Environment: WebSphere Application Server 7.0 with AIX 7.1 and JDK 1.6.0 > fop.jar, Implementation-Version: 1.1, Build-Id: 20121016-224756-ICT (glenn [Mac OS X 10.8.1 x86_64, Java 1.6.0_35-b10-428-11M3811, Target Java 1.5]) > xmlgraphics-commons-1.5.jar, Implementation-Version: 1.5, Build-Id: 20121016-215324-ICT (glenn [Mac OS X 10.8.1 x86_64, Java 1.6.0_35-b10-428-11M3811]) > Reporter: Andre Klemann > Labels: easyfix, performance > Fix For: trunk > > Attachments: xgc-93.patch > > > The method org.apache.xmlgraphics.ps.ImageEncodingHelper.optimizedWriteTo(OutputStream) from xmlgraphics-commons-1.5.jar calls the method DeflaterOutputStream.write(byte) byte by byte (see lines 242 - 244): > private boolean optimizedWriteTo(OutputStream out) > throws IOException { > if (this.firstTileDump) { > Raster raster = image.getTile(0, 0); > DataBuffer buffer = raster.getDataBuffer(); > if (buffer instanceof DataBufferByte) { > byte[] bytes = ((DataBufferByte) buffer).getData(); > // see determineEncodingColorModel() to see why we permute B and R here > if (isBGR) { > for (int i = 0; i < bytes.length; i += 3) { > out.write(bytes[i + 2]); // 242 > out.write(bytes[i + 1]); // 243 > out.write(bytes[i]); // 244 > } > } else { > out.write(bytes); > } > return true; > } > } > return false; > } > Under WebSphere, in a multi-session environment, this leads to warnings about possibly hanging threads: > [07.03.13 08:34:22:673 MEZ] 0000000d ThreadMonitor W WSVR0605W: Thread "WebContainer : 1303" (00000547) has been active for 165445 milliseconds and may be hung. There is/are 1 thread(s) in total in the server that may be hung. > at java.util.zip.Deflater.deflateBytes(Native Method) > at java.util.zip.Deflater.deflate(Deflater.java:306) > at java.util.zip.DeflaterOutputStream.deflate(DeflaterOutputStream.java:153) > at java.util.zip.DeflaterOutputStream.write(DeflaterOutputStream.java:112) > at java.util.zip.DeflaterOutputStream.write(DeflaterOutputStream.java:89) > at org.apache.xmlgraphics.ps.ImageEncodingHelper.optimizedWriteTo(ImageEncodingHelper.java:242) > ... > [07.03.13 08:38:18:886 MEZ] 00000547 ThreadMonitor W WSVR0606W: Thread "WebContainer : 1303" (00000547) was previously reported to be hung but has completed. It was active for approximately 401662 milliseconds. There is/are 0 thread(s) in total in the server that still may be hung. > [07.03.13 08:40:22:723 MEZ] 0000002d ThreadMonitor W WSVR0605W: Thread "WebContainer : 1323" (0000055b) has been active for 221920 milliseconds and may be hung. There is/are 1 thread(s) in total in the server that may be hung. > at java.util.zip.Deflater.deflateBytes(Native Method) > at java.util.zip.Deflater.deflate(Deflater.java:306) > at java.util.zip.DeflaterOutputStream.deflate(DeflaterOutputStream.java:153) > at java.util.zip.DeflaterOutputStream.write(DeflaterOutputStream.java:112) > at java.util.zip.DeflaterOutputStream.write(DeflaterOutputStream.java:89) > at org.apache.xmlgraphics.ps.ImageEncodingHelper.optimizedWriteTo(ImageEncodingHelper.java:244) > ... > The threads will finish their work, but this takes a long time. > The reason for this is, that the number n of calls to java.util.zip.DeflaterOutputStream.write(byte) might be in a hight range. I have seen cases, where n=4094496. This leads to n calls of DeflaterOutputSteam.write(byte[] b, int off, int len), where off=0 and len=1: > public void write(byte[] b, int off, int len) throws IOException { > if (def.finished()) { > throw new IOException("write beyond end of stream"); > } > if ((off | len | (off + len) | (b.length - (off + len))) < 0) { > throw new IndexOutOfBoundsException(); > } else if (len == 0) { > return; > } > if (!def.finished()) { > // Deflate no more than stride bytes at a time. This avoids > // excess copying in deflateBytes (see Deflater.c) > int stride = buf.length; > for (int i = 0; i < len; i+= stride) { > def.setInput(b, off + i, Math.min(stride, len - i)); // line 116 > while (!def.needsInput()) { > deflate(); // line 118 > } > } > } > } > Since java.util.zip.Deflater internally uses synchronized blocks, esspecially in setInput(byte[], int, int) and deflate(byte[], int, int), one should reduce the number of calls to these methods as much as possible. > The number of calls of Deflater.setInput(byte[] b, int off, int len) in line 116 > above could, e.g., be reduced by the factor 512 (=buffer.length), if the method ImageEncodingHelper.optimizedWriteTo() would be written in this way: > private boolean optimizedWriteTo(OutputStream out) > throws IOException { > if (this.firstTileDump) { > Raster raster = image.getTile(0, 0); > DataBuffer buffer = raster.getDataBuffer(); > if (buffer instanceof DataBufferByte) { > byte[] bytes = ((DataBufferByte) buffer).getData(); > // see determineEncodingColorModel() to see why we permute B and R here > if (isBGR) { > byte[] bytesPermutated = new byte[bytes.length]; > for (int i = 0; i < bytes.length; i += 3) { > bytesPermutated[i] = bytes[i+2]; > bytesPermutated[i+1] = bytes[i+1]; > bytesPermutated[i+2] = bytes[i]; > } > out.write(bytesPermutated); > } else { > out.write(bytes); > } > return true; > } > } > return false; > } > In this version, n calls of DeflaterOutputStream.write(byte) are replaced by one call of DeflaterOutputStream(byte[]), which is much faster and offers less blocking potential. > Without this optimization and with activated zlib compression in FOP 1.1, we always got warnings from WeSphere's thread monitor, that some threads may hang. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: general-unsubscribe@xmlgraphics.apache.org For additional commands, e-mail: general-help@xmlgraphics.apache.org