From dev-return-31115-apmail-forrest-dev-archive=forrest.apache.org@forrest.apache.org Tue May 18 00:47:07 2010 Return-Path: Delivered-To: apmail-forrest-dev-archive@www.apache.org Received: (qmail 61714 invoked from network); 18 May 2010 00:47:07 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 18 May 2010 00:47:07 -0000 Received: (qmail 80012 invoked by uid 500); 18 May 2010 00:47:07 -0000 Delivered-To: apmail-forrest-dev-archive@forrest.apache.org Received: (qmail 79967 invoked by uid 500); 18 May 2010 00:47:07 -0000 Mailing-List: contact dev-help@forrest.apache.org; run by ezmlm Precedence: bulk list-help: list-unsubscribe: List-Post: Reply-To: dev@forrest.apache.org List-Id: Delivered-To: mailing list dev@forrest.apache.org Received: (qmail 79960 invoked by uid 99); 18 May 2010 00:47:07 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 18 May 2010 00:47:07 +0000 X-ASF-Spam-Status: No, hits=0.7 required=10.0 tests=MIME_QP_LONG_LINE,RCVD_IN_DNSWL_NONE,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (nike.apache.org: local policy) Received: from [81.169.146.182] (HELO mo-p05-ob.rzone.de) (81.169.146.182) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 18 May 2010 00:46:57 +0000 X-RZG-AUTH: :I3wBe1Onbu02NkHlDbrpfe8kNG+qAh4Y0eq1FQR3dyEi7cXgY6D7Lf2OCmU8gGlZzxY= X-RZG-CLASS-ID: mo05 Received: from [192.168.1.33] (135.Red-88-19-61.staticIP.rima-tde.net [88.19.61.135]) by post.strato.de (jimi mo40) (RZmta 23.2) with (AES128-SHA encrypted) ESMTP id L002f6m4HKwk8t for ; Tue, 18 May 2010 02:46:36 +0200 (MEST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Apple Message framework v1078) Subject: Re: svn commit: r945269 - in /forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispatcher/src: cocoon-2.2-block/src/main/resources/COB-INF/ cocoon-2.2-block/src/main/resources/META-INF/cocoon/avalon/ java/org/apache/forrest/dispatc From: Thorsten Scherler In-Reply-To: Date: Tue, 18 May 2010 02:46:34 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <23696502-1453-4336-9DB4-BC70A26BB2C1@apache.org> References: To: dev@forrest.apache.org X-Mailer: Apple Mail (2.1078) X-Virus-Checked: Checked by ClamAV on apache.org On 18/05/2010, at 01:59, Thorsten Scherler wrote: >=20 > On 17/05/2010, at 23:24, Tim Williams wrote: >=20 >> Hi Thorsten, is this working for you locally? I noticed the >> Forrestbot failure just now and I'm also failing locally (with a NPE) >> on a dispatcher sample site. Was thinking it might be related but >> don't see anything obvious in here... >=20 > Hmm, really weird. I did a quick debug and it seems the problem is in=20= > > If you comment this line then it will work again. >=20 > I ATM not sure what happens and am flying out in a couple of hours, I = will try to have a look again tomorrow but if you want you can revert = the commit or comment the contract in the structurer. The problem seems to be=20 However I am not sure, need to go to bed now. salu2 >=20 > salu2 >=20 >>=20 >> --tim >>=20 >> On Mon, May 17, 2010 at 1:42 PM, wrote: >>> Author: thorsten >>> Date: Mon May 17 17:42:05 2010 >>> New Revision: 945269 >>>=20 >>> URL: http://svn.apache.org/viewvc?rev=3D945269&view=3Drev >>> Log: >>> FOR-1194 Fixing utf-8 compability by forcing to use UTF-8 in every = step >>>=20 >>> Modified: >>> = forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispat= cher/src/cocoon-2.2-block/src/main/resources/COB-INF/sitemap.xmap >>> = forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispat= cher/src/cocoon-2.2-block/src/main/resources/META-INF/cocoon/avalon/dispat= cher-sitemapcomponents.xconf >>> = forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispat= cher/src/java/org/apache/forrest/dispatcher/impl/XSLContract.java >>> = forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispat= cher/src/java/org/apache/forrest/dispatcher/impl/helper/XSLContractHelper.= java >>> = forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispat= cher/src/java/org/apache/forrest/dispatcher/transformation/DispatcherTrans= former.java >>> = forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispat= cher/src/java/org/apache/forrest/dispatcher/transformation/DispatcherWrapp= erTransformer.java >>>=20 >>> Modified: = forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispat= cher/src/cocoon-2.2-block/src/main/resources/COB-INF/sitemap.xmap >>> URL: = http://svn.apache.org/viewvc/forrest/trunk/whiteboard/plugins/org.apache.f= orrest.plugin.internal.dispatcher/src/cocoon-2.2-block/src/main/resources/= COB-INF/sitemap.xmap?rev=3D945269&r1=3D945268&r2=3D945269&view=3Ddiff >>> = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D >>> --- = forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispat= cher/src/cocoon-2.2-block/src/main/resources/COB-INF/sitemap.xmap = (original) >>> +++ = forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispat= cher/src/cocoon-2.2-block/src/main/resources/COB-INF/sitemap.xmap Mon = May 17 17:42:05 2010 >>> @@ -22,7 +22,7 @@ xmlns:map=3D"http://apache.org/cocoon/site >>> >>> >>> >>> - >>> + >>> >>> >>> >>> @@ -61,7 +61,7 @@ xmlns:map=3D"http://apache.org/cocoon/site >>> >>> >>> >>> - >>> + >>> >>> >>> >>> @@ -93,11 +93,11 @@ xmlns:map=3D"http://apache.org/cocoon/site >>> >>> >>> >>> - >>> + >>> >>> >>> >>> - >>> + >>> >>> >>> >>> @@ -105,7 +105,7 @@ xmlns:map=3D"http://apache.org/cocoon/site >>> >>> >>> >>> - >>> + >>> >>> >>> >>> @@ -116,7 +116,7 @@ xmlns:map=3D"http://apache.org/cocoon/site >>> >>> >>> >>> - >>> + >>> >>> >>> >>> @@ -126,7 +126,7 @@ xmlns:map=3D"http://apache.org/cocoon/site >>> >>> >>> >>> - >>> + >>> >>> >>> >>>=20 >>> Modified: = forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispat= cher/src/cocoon-2.2-block/src/main/resources/META-INF/cocoon/avalon/dispat= cher-sitemapcomponents.xconf >>> URL: = http://svn.apache.org/viewvc/forrest/trunk/whiteboard/plugins/org.apache.f= orrest.plugin.internal.dispatcher/src/cocoon-2.2-block/src/main/resources/= META-INF/cocoon/avalon/dispatcher-sitemapcomponents.xconf?rev=3D945269&r1=3D= 945268&r2=3D945269&view=3Ddiff >>> = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D >>> --- = forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispat= cher/src/cocoon-2.2-block/src/main/resources/META-INF/cocoon/avalon/dispat= cher-sitemapcomponents.xconf (original) >>> +++ = forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispat= cher/src/cocoon-2.2-block/src/main/resources/META-INF/cocoon/avalon/dispat= cher-sitemapcomponents.xconf Mon May 17 17:42:05 2010 >>> @@ -28,7 +28,7 @@ limitations under the License. >>> true >>> >>> >>> - >>> + >>> >> mime-type=3D"text/html; charset=3DUTF-8" name=3D"xhtml" = ppool-max=3D"64" >>> src=3D"org.apache.cocoon.serialization.XMLSerializer"> >>> @@ -57,5 +57,9 @@ limitations under the License. >>> UTF-8 >>> true >>> >>> - >>> + >> + src=3D"org.apache.cocoon.serialization.XMLSerializer"> >>> + UTF-8 >>> + >>> + >>> >>>=20 >>> Modified: = forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispat= cher/src/java/org/apache/forrest/dispatcher/impl/XSLContract.java >>> URL: = http://svn.apache.org/viewvc/forrest/trunk/whiteboard/plugins/org.apache.f= orrest.plugin.internal.dispatcher/src/java/org/apache/forrest/dispatcher/i= mpl/XSLContract.java?rev=3D945269&r1=3D945268&r2=3D945269&view=3Ddiff >>> = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D >>> --- = forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispat= cher/src/java/org/apache/forrest/dispatcher/impl/XSLContract.java = (original) >>> +++ = forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispat= cher/src/java/org/apache/forrest/dispatcher/impl/XSLContract.java Mon = May 17 17:42:05 2010 >>> @@ -21,6 +21,8 @@ import java.io.BufferedOutputStream; >>> import java.io.ByteArrayInputStream; >>> import java.io.ByteArrayOutputStream; >>> import java.io.InputStream; >>> +import java.io.OutputStreamWriter; >>> +import java.io.UnsupportedEncodingException; >>> import java.util.Map; >>>=20 >>> import javax.xml.stream.XMLStreamException; >>> @@ -105,17 +107,23 @@ public class XSLContract extends Loggabl >>> } >>> } >>> ByteArrayOutputStream out =3D new ByteArrayOutputStream(); >>> + byte[] bytes =3D null; >>> + String utf8 =3D ""; >>> // create a StreamResult and use it for the transformation >>> - Result streamResult =3D new StreamResult(new = BufferedOutputStream(out)); >>> try { >>> - helper.transform(dataStream,streamResult); >>> + OutputStreamWriter writer =3D new = OutputStreamWriter(out,"UTF-8"); >>> + Result streamResult =3D new StreamResult(writer); >>> + helper.transform(dataStream,streamResult); >>> + utf8 =3D out.toString("UTF-8"); >>> + log.debug(utf8); >>> + bytes =3D utf8.getBytes("utf-8"); >>> } catch (Exception e) { >>> String message =3D "Could not invoke the transformation for " >>> + "the contract \""+name+"\". "+"\n"+ e; >>> throw new ContractException(message); >>> } >>> - log.debug(out.toString()); >>> - return new BufferedInputStream(new = ByteArrayInputStream(out.toByteArray())); >>> + >>> + return new BufferedInputStream(new = ByteArrayInputStream(bytes)); >>> } >>>=20 >>> /* (non-Javadoc) >>>=20 >>> Modified: = forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispat= cher/src/java/org/apache/forrest/dispatcher/impl/helper/XSLContractHelper.= java >>> URL: = http://svn.apache.org/viewvc/forrest/trunk/whiteboard/plugins/org.apache.f= orrest.plugin.internal.dispatcher/src/java/org/apache/forrest/dispatcher/i= mpl/helper/XSLContractHelper.java?rev=3D945269&r1=3D945268&r2=3D945269&vie= w=3Ddiff >>> = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D >>> --- = forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispat= cher/src/java/org/apache/forrest/dispatcher/impl/helper/XSLContractHelper.= java (original) >>> +++ = forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispat= cher/src/java/org/apache/forrest/dispatcher/impl/helper/XSLContractHelper.= java Mon May 17 17:42:05 2010 >>> @@ -20,6 +20,7 @@ import java.io.BufferedInputStream; >>> import java.io.ByteArrayInputStream; >>> import java.io.IOException; >>> import java.io.InputStream; >>> +import java.io.InputStreamReader; >>> import java.io.PrintWriter; >>> import java.util.Iterator; >>> import java.util.Map; >>> @@ -124,6 +125,7 @@ public class XSLContractHelper extends L >>> SAXException, IOException { >>> // prepare transformation >>> transformer =3D transFact.newTransformer(xslSource); >>> + transformer.setOutputProperty(OutputKeys.ENCODING, "UTF-8"); >>> transformer.setOutputProperty(OutputKeys.OMIT_XML_DECLARATION, = "yes"); >>> transformer.setOutputProperty(OutputKeys.INDENT, "yes"); >>> transformer.setOutputProperty(OutputKeys.METHOD, "xml"); >>> @@ -220,8 +222,8 @@ public class XSLContractHelper extends L >>> } >>> // Setting the necessary attributes of the contract >>> // xsl >>> - contract.setXslSource(new StreamSource(new = ByteArrayInputStream(handler >>> - .getBytes()))); >>> + byte[] bytes =3D handler.getBytes(); >>> + contract.setXslSource(new StreamSource(new = ByteArrayInputStream(bytes))); >>> /* DEBUG_CODE: >>> * The following is useful to see what is going on >>> * >>> @@ -254,7 +256,9 @@ public class XSLContractHelper extends L >>> throws ContractException { >>> //Source dataSource =3D new StreamSource(dataStream); >>> try { >>> - SAXSource saxSource =3D new SAXSource(xmlReader,new = InputSource(dataStream)); >>> + InputSource inputSource =3D new InputSource(new = InputStreamReader(dataStream, "UTF-8")); >>> + inputSource.setEncoding("UTF-8"); >>> + SAXSource saxSource =3D new SAXSource(xmlReader,inputSource); >>> transformer.transform(saxSource, streamResult); >>> } catch (Exception e) { >>> String message =3D "The xsl transformation has thrown an = exception. for " >>>=20 >>> Modified: = forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispat= cher/src/java/org/apache/forrest/dispatcher/transformation/DispatcherTrans= former.java >>> URL: = http://svn.apache.org/viewvc/forrest/trunk/whiteboard/plugins/org.apache.f= orrest.plugin.internal.dispatcher/src/java/org/apache/forrest/dispatcher/t= ransformation/DispatcherTransformer.java?rev=3D945269&r1=3D945268&r2=3D945= 269&view=3Ddiff >>> = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D >>> --- = forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispat= cher/src/java/org/apache/forrest/dispatcher/transformation/DispatcherTrans= former.java (original) >>> +++ = forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispat= cher/src/java/org/apache/forrest/dispatcher/transformation/DispatcherTrans= former.java Mon May 17 17:42:05 2010 >>> @@ -20,6 +20,7 @@ import java.io.BufferedInputStream; >>> import java.io.File; >>> import java.io.IOException; >>> import java.io.InputStream; >>> +import java.io.InputStreamReader; >>> import java.io.Serializable; >>> import java.io.StringReader; >>> import java.util.HashMap; >>> @@ -490,7 +491,7 @@ public class DispatcherTransformer exten >>> throw new SAXException(e); >>> } >>> if (null !=3D property) { >>> - localParams.put(currentProperty, = property.getBytes()); >>> + localParams.put(currentProperty, = property.getBytes("UTF-8")); >>> } >>> } >>> } >>> @@ -695,7 +696,7 @@ public class DispatcherTransformer exten >>> root.serialize(out); >>> } >>>=20 >>> - InputSource is =3D new InputSource(new = StringReader(out.toString())); >>> + InputSource is =3D new InputSource(new = StringReader(out.toString("UTF-8"))); >>> // adding the result to the consumer >>> parser.parse(is, super.xmlConsumer); >>> } catch (Exception e) { >>> @@ -1099,8 +1100,9 @@ public class DispatcherTransformer exten >>> */ >>> private static InputSource getInputSource(final Source source) >>> throws IOException, SourceException { >>> - final InputSource newObject =3D new = InputSource(source.getInputStream()); >>> - newObject.setSystemId(source.getURI()); >>> - return newObject; >>> + final InputSource inputSource =3D new InputSource(new = InputStreamReader(source.getInputStream(), "UTF-8")); >>> + inputSource.setEncoding("UTF-8"); >>> + inputSource.setSystemId(source.getURI()); >>> + return inputSource; >>> } >>> } >>>=20 >>> Modified: = forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispat= cher/src/java/org/apache/forrest/dispatcher/transformation/DispatcherWrapp= erTransformer.java >>> URL: = http://svn.apache.org/viewvc/forrest/trunk/whiteboard/plugins/org.apache.f= orrest.plugin.internal.dispatcher/src/java/org/apache/forrest/dispatcher/t= ransformation/DispatcherWrapperTransformer.java?rev=3D945269&r1=3D945268&r= 2=3D945269&view=3Ddiff >>> = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D >>> --- = forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispat= cher/src/java/org/apache/forrest/dispatcher/transformation/DispatcherWrapp= erTransformer.java (original) >>> +++ = forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.internal.dispat= cher/src/java/org/apache/forrest/dispatcher/transformation/DispatcherWrapp= erTransformer.java Mon May 17 17:42:05 2010 >>> @@ -374,7 +374,7 @@ public class DispatcherWrapperTransforme >>> } >>> // get the result of the structurer as stream >>> InputStream result =3D structurer.execute(new = BufferedInputStream( >>> - new ByteArrayInputStream(document.getBytes())), = requestedFormat); >>> + new ByteArrayInputStream(document.getBytes("UTF-8"))), = requestedFormat); >>> // requesting a parser >>> parser =3D (SAXParser) manager.lookup(SAXParser.ROLE); >>> // adding the result to the consumer >>>=20 >>>=20 >>>=20 >=20 > Thorsten Scherler > Open Source Java >=20 Thorsten Scherler Open Source Java