avro-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kevin Schultz (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AVRO-1493) Avoid the "Turkish Locale Problem"
Date Wed, 09 Dec 2015 15:52:11 GMT

    [ https://issues.apache.org/jira/browse/AVRO-1493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15048863#comment-15048863
] 

Kevin Schultz commented on AVRO-1493:
-------------------------------------

We are seeing the same error using Avro on Android. I dug around a bit and at least for the
fingerprint generation problem my guess is that the problem is caused by a few places where
the Locale is not explicitly set. 

org.apache.avro.generic.GenericData.java:      builder.append(hex.toUpperCase());
org.apache.avro.Schema.java:                          order Field.Order.valueOf(orderNode.getTextValue().toUpperCase());
org.apache.avro.Schema.java:                          private Type() { this.name = this.name().toLowerCase();
}
org.apache.avro.Schema.java:                          private Order() { this.name = this.name().toLowerCase();
}



> Avoid the "Turkish Locale Problem"
> ----------------------------------
>
>                 Key: AVRO-1493
>                 URL: https://issues.apache.org/jira/browse/AVRO-1493
>             Project: Avro
>          Issue Type: Bug
>          Components: java
>    Affects Versions: 1.7.6
>         Environment: Hadoop trunk build error on mac-os with turkish locale.
>            Reporter: Serkan Taş
>             Fix For: 1.7.8
>
>
> Locale dependent String.toUpperCase(), String.toLowerCase() causes unexpected behavior
if the the locale is Turkish
> Not sure about String.equalsIgnoreCase(..).
> Here is the error :
> [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.5.1:testCompile
(default-testCompile) on project hadoop-common: Compilation failure
> [ERROR] /Users/serkan/programlar/dev/hadooptest/hadoop-trunk/hadoop-common-project/hadoop-common/target/generated-test-sources/java/org/apache/hadoop/io/serializer/avro/AvroRecord.java:[10,244]
unmappable character for encoding UTF-8
> [ERROR] -> [Help 1]
> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.5.1:testCompile
(default-testCompile) on project hadoop-common: Compilation failure
> /Users/serkan/programlar/dev/hadooptest/hadoop-trunk/hadoop-common-project/hadoop-common/target/generated-test-sources/java/org/apache/hadoop/io/serializer/avro/AvroRecord.java:[10,244]
unmappable character for encoding UTF-8
> I f i check the code i discovered the reason for error :
>  public static final org.apache.avro.Schema SCHEMA$ = new org.apache.avro.Schema.Parser().parse("{\"type\":\"record\",\"name\":\"AvroRecord\",\"namespace\":\"org.apache.hadoop.io.serializer.avro\",\"fields\":[{\"name\":\"intField\",\"type\":\"Ýnt\"}]}");
> For the code generated from schema, locale dependent capitalization of letter "i" turns
in to "Ý" should be the same for "I" to "ı".
> Same bug exist in OPENEJB-1071, OAK-260, IBATIS-218.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message