camel-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Siano, Stephan" <>
Subject RE: Camel build, tests and charsets
Date Mon, 02 May 2016 05:41:40 GMT
Hi Claus,

Ok, this solves the string definition issue, however the platform encoding issue is somewhat
more difficult. For the specific use case I might find a way around the issue, but in general
we should really think about changing the build to some other platform encoding to find encoding
issues better. In the current setup both the platform encoding and the default encoding for
camel exchanges are UTF-8 (which contains all characters), so we will not automatically find
any issues where the platform default encoding is used instead of the encoding defined in
the camel exchange.

One  other question: I created CAMEL-9933 for the issue and assigned it to me (I guess this
is an easy enough fix for my first commit). How do I proceed in order to not disturb the camel
2.17.1 release? The fix is really simple and non-intrusive, but I don't think it is important
enough to block anything (and I will likely be slow since this is my first commit in camel).

Best regards

-----Original Message-----
From: Claus Ibsen [] 
Sent: Freitag, 29. April 2016 15:09
To: dev <>
Subject: Re: Camel build, tests and charsets

Is it "Lucky Luke" that has non ASCII chars?

You can use \uNNNN to specify non ascii chars. We do that in some unit tests.
I remember we have some for german, and also for the Thai elephant
which is named Chang, which is a good beer btw ;)

On Fri, Apr 29, 2016 at 2:49 PM, Siano, Stephan <> wrote:
> Hi,
> I have found an issue with the camel-csv component. The fix itself is pretty trivial
(a one-liner to use the IOHelper.getCharsetName(exchange)) for an OutputStreamWriter in the
marhaller (the InputStreamWriter in the unmarshaller already does this).
> However I have some difficulties with the unit test.
> 1.       The issue will only happen if there are non-ASCII characters in the code. As
far as I understand it is not exactly desired to have those in the coding (on the other hand
I just copied the appropriate test for the unmarshaller and that one does also have german
umlaut characters in the code.
> 2.       The test only fails (with the original code) if the default encoding on the
JVM that executes this is not UTF-8. There is some very hacky way to change the JVM default
encoding for a test but I would not want to contribute that.
> I put the test into this mail as an example (because it is short enough to demonstrate
the issues). How could I test this properly? Wouldn't it be a good idea in general to run
tests with an ASCII default charset to find issues with default encodings better (set file.encoding
to "ASCII" in the surefire plugin configuration)?
> Best regards
> Stephan
> public class CsvMarshalCharsetTest extends CamelTestSupport {
>     @BeforeClass
>     public static void hackEncoding() throws NoSuchFieldException, SecurityException,
IllegalArgumentException, IllegalAccessException {
>         System.setProperty("file.encoding", "ASCII");
>         Field charset = Charset.class.getDeclaredField("defaultCharset");
>         charset.setAccessible(true);
>         charset.set(null, null);
>     }
>     @Test
>     public void testMarshal() throws Exception {
>         MockEndpoint endpoint = getMockEndpoint("mock:daltons");
>         endpoint.expectedMessageCount(1);
>         List<List<String>> data = new ArrayList<List<String>>();
>         data.add(0, new ArrayList<String>());
>         data.get(0).add(0, "Lücky Luke");
>         template.sendBody("direct:start", data);
>         endpoint.assertIsSatisfied();
>         Exchange exchange = endpoint.getExchanges().get(0);
>         String body = exchange.getIn().getBody(String.class);
>         assertThat(body, startsWith("Lücky Luke"));
>     }
>     protected RouteBuilder createRouteBuilder() {
>         return new RouteBuilder() {
>             public void configure() {
>                 from("direct:start").marshal().csv().to("mock:daltons");
>             }
>         };
>     }
> }

Claus Ibsen
----------------- @davsclaus
Camel in Action 2:
View raw message