camel-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Claus Ibsen <>
Subject Re: Camel build, tests and charsets
Date Mon, 02 May 2016 07:12:58 GMT
On Mon, May 2, 2016 at 7:41 AM, Siano, Stephan <> wrote:
> Hi Claus,
> Ok, this solves the string definition issue, however the platform encoding issue is somewhat
more difficult. For the specific use case I might find a way around the issue, but in general
we should really think about changing the build to some other platform encoding to find encoding
issues better. In the current setup both the platform encoding and the default encoding for
camel exchanges are UTF-8 (which contains all characters), so we will not automatically find
any issues where the platform default encoding is used instead of the encoding defined in
the camel exchange.
> One  other question: I created CAMEL-9933 for the issue and assigned it to me (I guess
this is an easy enough fix for my first commit). How do I proceed in order to not disturb
the camel 2.17.1 release? The fix is really simple and non-intrusive, but I don't think it
is important enough to block anything (and I will likely be slow since this is my first commit
in camel).

You can backport to the 2.17.x branch.

Gregor has created the tag for 2.17.1

And in JIRA you can mark it as fixed for 2.17.2.

Also it may be good to backport to 2.16.x as well. As that branch is
still supported.

> Best regards
> Stephan
> -----Original Message-----
> From: Claus Ibsen []
> Sent: Freitag, 29. April 2016 15:09
> To: dev <>
> Subject: Re: Camel build, tests and charsets
> Is it "Lucky Luke" that has non ASCII chars?
> You can use \uNNNN to specify non ascii chars. We do that in some unit tests.
> I remember we have some for german, and also for the Thai elephant
> which is named Chang, which is a good beer btw ;)
> On Fri, Apr 29, 2016 at 2:49 PM, Siano, Stephan <> wrote:
>> Hi,
>> I have found an issue with the camel-csv component. The fix itself is pretty trivial
(a one-liner to use the IOHelper.getCharsetName(exchange)) for an OutputStreamWriter in the
marhaller (the InputStreamWriter in the unmarshaller already does this).
>> However I have some difficulties with the unit test.
>> 1.       The issue will only happen if there are non-ASCII characters in the code.
As far as I understand it is not exactly desired to have those in the coding (on the other
hand I just copied the appropriate test for the unmarshaller and that one does also have german
umlaut characters in the code.
>> 2.       The test only fails (with the original code) if the default encoding on
the JVM that executes this is not UTF-8. There is some very hacky way to change the JVM default
encoding for a test but I would not want to contribute that.
>> I put the test into this mail as an example (because it is short enough to demonstrate
the issues). How could I test this properly? Wouldn't it be a good idea in general to run
tests with an ASCII default charset to find issues with default encodings better (set file.encoding
to "ASCII" in the surefire plugin configuration)?
>> Best regards
>> Stephan
>> public class CsvMarshalCharsetTest extends CamelTestSupport {
>>     @BeforeClass
>>     public static void hackEncoding() throws NoSuchFieldException, SecurityException,
IllegalArgumentException, IllegalAccessException {
>>         System.setProperty("file.encoding", "ASCII");
>>         Field charset = Charset.class.getDeclaredField("defaultCharset");
>>         charset.setAccessible(true);
>>         charset.set(null, null);
>>     }
>>     @Test
>>     public void testMarshal() throws Exception {
>>         MockEndpoint endpoint = getMockEndpoint("mock:daltons");
>>         endpoint.expectedMessageCount(1);
>>         List<List<String>> data = new ArrayList<List<String>>();
>>         data.add(0, new ArrayList<String>());
>>         data.get(0).add(0, "Lücky Luke");
>>         template.sendBody("direct:start", data);
>>         endpoint.assertIsSatisfied();
>>         Exchange exchange = endpoint.getExchanges().get(0);
>>         String body = exchange.getIn().getBody(String.class);
>>         assertThat(body, startsWith("Lücky Luke"));
>>     }
>>     protected RouteBuilder createRouteBuilder() {
>>         return new RouteBuilder() {
>>             public void configure() {
>>                 from("direct:start").marshal().csv().to("mock:daltons");
>>             }
>>         };
>>     }
>> }
> --
> Claus Ibsen
> -----------------
> @davsclaus
> Camel in Action 2:

Claus Ibsen
----------------- @davsclaus
Camel in Action 2:

View raw message