db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Knut Anders Hatlen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DERBY-5958) Many tests fail on JDK 8 because of missing collators
Date Tue, 23 Oct 2012 14:03:13 GMT

    [ https://issues.apache.org/jira/browse/DERBY-5958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13482332#comment-13482332
] 

Knut Anders Hatlen commented on DERBY-5958:
-------------------------------------------

Looking more closely at this issue, I'm not sure if this is just a
test issue, or if it would be better to fix it in the engine code that
checks if a territory is supported. Especially, since many common
locales, like de_DE and fr_FR, are not in the array returned by the
JDK 8 version of Collator.getAvailableLocales(), being more liberal
with the locales we accept when booting a database with
territory-based collation, will be more helpful for existing users who
are upgrading.

The good news is that Collator.getInstance(Locale) still returns the
exact same collator as in Java 7, even for the locales that have been
removed from Collator.getAvailableLocales().

For example,

  Collator.getInstance(new Locale("fr", "BE")),
  Collator.getInstance(new Locale("fr", "CA")),
  Collator.getInstance(new Locale("fr", "CH")),
  Collator.getInstance(new Locale("fr", "FR")),
  Collator.getInstance(new Locale("fr", "LU"))

all return collators that work the same way as

  Collator.getInstance(new Locale("fr"))

even though only the latter is explicitly mentioned by
getAvailableLocales().

Similarly, even though "no_NO" is removed from the list, which caused
problems in the collation tests, Collator.getInstance(new Locale("no", "NO"))
still works the same way as Collator.getInstance(new Locale("no")).

Other locales used in the test, like "nl_NL" and "de_DE", don't have a
corresponding language-only ("nl" or "de") locale in
getAvailableLocales(). However, for these locales, both JDK 7 and JDK
8 return the same collator as they return for Locale.US, so there's no
actual change.

For completeness, here's what java.text.Collator.getAvailableLocales()
returns on JDK 8 (build 1.8.0-ea-b61):

ro
iw
mk
sv
sr
bg
uk
hi
hr
el
hu
vi
ar
th
lv
zh_HK
fr
en_US
tr
sq
zh
sk
no
lt
pl
cs
is
zh_TW
ja
es
sl
da
ko
et
fi
sr__#Latn
be
ca
ru

Only 39 values, whereas Java 7 returned 156 values.

It looks to me as if the pattern of the trimming is:

1) If the different variants of a language don't actually vary, only
   return the generic locale for that language and leave the country
   unspecified (example: the many French variants mentioned above are
   now represented by a single entry equal to Locale.FRENCH)

2) Those locales that use the same collator as the default "en_US"
   collator are removed (example: "de_DE", "nl_NL", "en_GB" all use
   the same collator as "en_US", so they are not among the returned
   locales)

The first one seems fairly straightforward to handle. When you boot a
database with territory fr_CA and territory-based collation, you first
check for the fully qualified locale in the array returned by
getAvailableLocales(), and if it's not there, check for the locale
without the country, that is, "fr".

The second case is trickier. If you boot a database with territory
de_DE and territory-based collation, you won't find "de_DE" or "de" in
the list, so the default collation will be used instead. However, you
don't know if the lack of "de_DE" and "de" in getAvailableLocales() is
because using the default en_US collation is correct for the locale or
if it is because the JVM doesn't have the special rules for the
locale, so there's not enough information to decide whether or not to
allow the boot to proceed.
                
> Many tests fail on JDK 8 because of missing collators
> -----------------------------------------------------
>
>                 Key: DERBY-5958
>                 URL: https://issues.apache.org/jira/browse/DERBY-5958
>             Project: Derby
>          Issue Type: Bug
>          Components: Test
>    Affects Versions: 10.10.0.0
>            Reporter: Knut Anders Hatlen
>
> Many of the regression tests fail on JDK 8 after http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7171324
trimmed down the list of locales returned by Collator.getAvailableLocales(). Many test cases
assume that certain locales are always available, even though the specification for Collator.getAvailableLocales()
only guarantees that Locale.US is available. We should add guards to the collation-dependent
test cases to prevent them from running when the required locale is not available in the test
environment.
> The most common source of the failures in my environment is the lack of collator support
for the 'en' locale, with stack traces like this:
> Caused by: ERROR XBM04: Collator support not available from the JVM for the database's
locale 'en'.
> 	at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
> 	at org.apache.derby.iapi.types.DataValueFactoryImpl.verifyCollatorSupport(Unknown Source)
> 	at org.apache.derby.iapi.types.DataValueFactoryImpl.boot(Unknown Source)
> 	at org.apache.derby.iapi.types.J2SEDataValueFactory.boot(Unknown Source)
> 	at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
> 	at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
> 	at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
> 	at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
> 	at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
> 	at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
> 	at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
> 	at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
> 	at org.apache.derby.impl.services.monitor.BaseMonitor.createPersistentService(Unknown
Source)
> 	at org.apache.derby.iapi.services.monitor.Monitor.createPersistentService(Unknown Source)
> Those should be easy enough to fix by making the tests use 'en_US' for territory, since
support for that locale is required by the Java specification.
> There are also other problems, like CollationTest2.suite(), which checks for the availability
for the "no" locale, and if it is available, it runs the test with territory=no_NO. This does
not work in an environment where the "no" locale is supported and the "no_NO" locale isn't.
We should change the availability check and the actual database territory to match.
> Another instance can be seen in tools/dblook_test_territory.java, which fails if a collator
for the nl_NL locale isn't available. Since this test runs in the old test harness, which
is less flexible when it comes to excluding tests based on the environment, it might be best
to convert it to JUnit first.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message