harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ivanov, Alexey A" <alexey.a.iva...@intel.com>
Subject RE: [testing] locale dependent tests
Date Tue, 25 Jul 2006 07:46:39 GMT

>-----Original Message-----
>From: Richard Liang [mailto:richard.liangyx@gmail.com]
>Sent: Tuesday, July 25, 2006 6:26 AM
>To: harmony-dev@incubator.apache.org
>Subject: Re: [testing] locale dependent tests
>
>
>
>Ivanov, Alexey A wrote:
>>
>>> -----Original Message-----
>>> From: Paulex Yang [mailto:paulex.yang@gmail.com]
>>> Sent: Thursday, July 20, 2006 10:27 AM
>>> To: harmony-dev@incubator.apache.org
>>> Subject: Re: [testing] locale dependent tests
>>>
>>> Andrew Zhang wrote:
>>>
>>>> On 7/19/06, Richard Liang <richard.liangyx@gmail.com> wrote:
>>>>
>>>>>
>>>>> Tim Ellison wrote:
>>>>>
>>>>>> Richard Liang wrote:
>>>>>>
>>>>>>
>>>>>>> Although the spec does not require the round-trip of
applyPattern
>>>>>>>
>> and
>>
>>>>>>> toPattern, we still want to get one *certain* pattern through
>>>>>>>
>>>>> toPattern.
>>>>>
>>>>>>> Now the problem is the returned pattern is locale-dependent.
I'm
>>>>>>> uncertain about the reason to remove the assertion:
>>>>>>> 1) Because the behavior is not required by spec, or
>>>>>>> 2) Because the behavior is locale-dependent
>>>>>>>
>>>>>>>
>>>>>> It would seem that rather than forcing the locale to be en_US to
>>>>>>
>> make
>>
>>>>>> the test assertions valid, you could work with the default locale
>>>>>>
>> and
>>
>>>>>> guard any assertions that are locale-specific.  It would seem a
>>>>>>
>>>>> shame to
>>>>>
>>>>>> loose the diversity of testing on multiple locale machines if the
>>>>>>
>>>>> tests
>>>>>
>>>>>> always force everyone to look like an American (horror! ;-) )
>>>>>>
>>>>>> I would expect the fix therefore to be something like, if locale
>>>>>>
>> is
>>
>>>>>> en_US go ahead and assert more round-tripping code, otherwise
>>>>>>
>> can't
>>
>>>>> test
>>>>>
>>>>>> it as the spec allows variances.
>>>>>>
>>>>>>
>>>>>>
>>>>> If a test case passes in all locales, could we think that the
>>>>>
>> behavior
>>
>>>>> tested by the test case is locale-independent? We still have to
>>>>>
>> think
>>
>>>>> about how to test locale-dependent behavior. For example,
>>>>> java.util.Locale.getDisplayName() and
>>>>>
>> java.lang.String.toUpperCase(). To
>>
>>>>> verify both the code logic and locale data, shall we have some
tests
>>>>> like ABC_en_US_Test, ABC_en_UK_Test, and ABC_ru_RU_Test?
>>>>>
>>> I still confuse what we want to test, the logic or the data? I think
>>> most (if not all) i18n related methods actually have same single
>>> executable with multiple resource bundles, i.e., the single
executable
>>> should be locale-independent, the different return value is due to
the
>>> resource data difference. I think at least for now, we should pay
our
>>> attention to logic of single executable, and leave the data
>>>
>> verification
>>
>>> to the i18n libraries' author, say, ICU, they have much more
knowledge
>>> and authority (at least than me) on this area.
>>>
>>> If we can get agree on the above, so the i18n related test cases
>>> organization are easier to judge: the logic is locale-independent,
so
>>> ideally the tests should be locale-independent, but we have some
>>> exceptional cases, say, the en_UK in MessageFormat case, so we
cannot
>>> make our tests rely on the default locale, then we just specify one
>>> locale(en_US) to the tests, and supplement some exceptional case
when
>>>
>> we
>>
>>> find some. i.e., I don't think we need ABC_en_US_Test, or so.
>>>
>>> Comments?
>>>
>>>
>>
>> I don't think we need several test cases like ABC_en_US_Test and
>> ABC_ru_RU_Test because users can modify locale data. Perhaps not all
>> data can be changed, but some can be surely, for example, date/time
>> formats, decimal and group separators. Thus a test which passes on
one
>> machine can fail on another one because locale data are different
from
>> the default values.
>>
>
>I agree that we do not need the test cases like ABC_en_US_Test and
>ABC_ru_RU_Test. But I'm just wondering whether users could modify the
>locale data. Would you please give some instructions? Thanks a lot.

In Windows XP, open the Control Panel, choose "Date, Time, Language, and
Regional Options" group and then click "Regional and Language Options"
(if in Classic View, just click this option). On the Regional Options
tab click Customize button. A dialog opens. Here one can change
formatting options like decimal separator symbol, digit grouping symbol,
date format, time format and others.

I can't give you instructions for Linux but surely there should be a way
to customize these data.

I know for sure that in Russia many users customize decimal separator to
be point like in English locale. This is because many programs had
problems dealing with something other than point in numbers. As far as I
know the situation is better now.

>
>> So, I think the best way to deal with such tests is to provide a
"fake"
>> hard-coded locale which can't be changed at all. And tests will
become
>> locale-independent.
>>
>Not sure what the "fake" hard-coded locale is. I used to set the
default
>locale to some particular one, e.g., en_US, but it seems that someone
>(Tim) does not like the idea. :-)
>

By "fake" locale I mean a locale that doesn't fetch any settings from
operating system. It may be en_US or another. However we should ensure
the data in this locale can't be changed.

If you had set en_US in the tests, we wouldn't have found bugs in
DecimalFormatter, for example HARMONY-965. All tests would have passed
successfully in Russian locale as well.

So we need to ensure that the logic is correct. How? It's a good
question. We could use non-standard data for a locale used for testing,
and this may help. However I believe we can't find all bugs this way
too.

App-driven Improvement proposed by Geir [1] can help with this.

As another option, we can fetch data we expect to be used from locale,
create strings to compare with using this data. [2] But this approach
will make tests almost unreadable.

Thanks,
Alexey.


[1]
http://mail-archives.apache.org/mod_mbox/incubator-harmony-dev/200607.mb
ox/%3c44C4204D.8090406@pobox.com%3e
[2]
http://mail-archives.apache.org/mod_mbox/incubator-harmony-dev/200607.mb
ox/%3c8E389A5F2FEABA4CB1DEC35A25CB39CE122A16@mssmsx411%3e

>>
>> Any thoughts?
>>
>>
>> Thanks,
>> Alexey.
>>
>>
>>>> Hi Richard,
>>>> For getDisplayName, getDisplayLanguage() and methods like so, which
>>>>
>> are
>>
>>>> locale-dependent, I suggest we write implementation tests for them.
>>>> The test
>>>> may look like:
>>>> 1. get default locale
>>>> 2. get i18n string from ResourceBundle directly
>>>> 3. get i18n string by locale-dependent method
>>>> 4. assertEquals
>>>>
>>> If we write test cases like this, these tests are probably
>>> locale-independent, because: the executable is probably single. I
don't
>>> think we should have many "locale-dependent" methods, we just have
many
>>> methods with "locale-dependent" data.
>>>
>>>> Sounds reasonable?
>>>>
>>>> Any comments? Thank you!
>>>>
>>>> Best regards,
>>>>
>>>>> Richard
>>>>>
>>>>>
>>>>>> Regards,
>>>>>> Tim
>>>>>>
>>>>>>
>>>>>>
>>>>> --
>>>>> Richard Liang
>>>>> China Software Development Lab, IBM
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>> --
>>> Paulex Yang
>>> China Software Development Lab
>>> IBM
>>>
>>>
>>>
>>>
---------------------------------------------------------------------
>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>> For additional commands, e-mail:
harmony-dev-help@incubator.apache.org
>>>
>>
>>
>> --
>> Alexey A. Ivanov
>> Intel Middleware Product Division
>>
>> ---------------------------------------------------------------------
>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>> For additional commands, e-mail:
harmony-dev-help@incubator.apache.org
>>
>>
>>
>
>--
>Richard Liang
>China Software Development Lab, IBM

--
Alexey A. Ivanov
Intel Middleware Product Division

---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org


Mime
View raw message