Mailing-List: contact dev-help@harmony.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@harmony.apache.org
Received-SPF: pass (athena.apache.org: domain of oliver.deakin@googlemail.com
 designates 209.85.219.158 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=googlemail.com; s=gamma;
        h=message-id:date:from:user-agent:mime-version:to:subject:references
         :in-reply-to:content-type:content-transfer-encoding;
        b=l2Lgfs1UfzsMREKmIQsYBu08Xeg+77ijdU2kEUI1Sy7DPhsIhzzxmShr/NSzep6bNn
         flf5+aNCFChm9nFBRwO5qMAkEWc3sL57CjXKV9nuw2tCGWz/44gOkoG9nAp67kJ6iXVF
         SOCI4n2b1ZxL3oRA3lb2NYuRAvxCfXE6397bI=
Message-ID: <4A152D4A.3040705@googlemail.com>
Date: Thu, 21 May 2009 11:30:34 +0100
From: Oliver Deakin <oliver.deakin@googlemail.com>
User-Agent: Thunderbird 2.0.0.21 (Windows/20090302)
MIME-Version: 1.0
To: dev@harmony.apache.org
Subject: Re: [classlib][build] Need depends.properties to be in ASCII on
 non-ASCII platforms
References: <4A141951.2030302@googlemail.com>
 <3b3f27c60905201828n29eb4160wbd0360abf1892e3@mail.gmail.com>
In-Reply-To: <3b3f27c60905201828n29eb4160wbd0360abf1892e3@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On z/OS, svn (or rather svnkit - the native svn client doesn't work on 
z/OS) automatically converts text/plain files into the native encoding 
at checkout, which is generally a good thing because we want at least 
the native code to be in native encoding. There is only one exception I 
have come across - the svn client seems to check the <?xml ... ?> header 
tag in xml files for an encoding property. If it is specified UTF-8 (as 
in our build files) svn leaves the files in the UTF-8. If the encoding 
is not specified, svn will convert the xml file to the native encoding.

I have also tested the rumoured svnkit:charset and svn:charset 
properties, but sadly they do not seem to work either. I'm starting to 
think that (2) from my original mail may be the best solution - although 
it's a little ugly, at least it only gets run at the fetch-depends stage.

Regards,
Oliver

Nathan Beyer wrote:
> Does SVN really do anything to the encoding of a text/plain file? I
> thought the only thing SVN did to files was swap EOL sequences. When
> using SVN across Win, Linux and Mac on a UTF-8 encoded file, I've
> always had the encoding maintained, even though each OS uses a
> different default encoding. I don't suppose this is a z/OS quirk, is
> it?
>
> On Wed, May 20, 2009 at 9:53 AM, Oliver Deakin
> <oliver.deakin@googlemail.com> wrote:
>   
>> Hi all,
>>
>> I have an issue building on z/OS - a fresh checkout from svn gives us
>> depends.properties in the default platform encoding (EBCDIC). However, when
>> we come to load these properties in depends-common.xml, using the
>> loadproperties Ant task, they are assumed to be in ASCII encoding. The
>> "encoding" property of the Ant task does not actually work correctly (I
>> think this is a bug in Ant which I intend to raise) so it is no help. I was
>> wondering if anybody has any ideas about the best way to tackle this
>> problem? Really I'd like to tag the file in some way so that it is checked
>> out of Subversion in the correct encoding immediately, but there does not
>> seem to be a way to do this. I have come up with a few options:
>>
>> 1) Mark the file binary in svn - not a good solution, means losing revision
>> history, not getting proper diffs etc.
>> 2) Use the Ant copy task with it's outputencoding property set to ISO-8859-1
>> to copy the file to another location, and then use that copy to load the
>> properties from. Delete the file after it has been used (patch would be like
>> [1]). Unfortunately this means that when we run the fetch-depends target we
>> see a lot of copy/delete output (setting verbose="false" for these tasks
>> does not seem to hide the output completely).
>> 3) Same as (2), but don't delete the file afterwards. There would only be
>> one initial copy, then no further copies unless the file is modified.
>> However, this does mean that these is an extra file left in the make
>> directory, which I don't like the idea of.
>>
>> I'd choose to go with (2), but thought I'd see if anyone has any other ideas
>> before I commit the change.
>>
>> Regards,
>> Oliver
>>
>>
>> [1]
>> Index: depends-common.xml
>> ===================================================================
>> --- depends-common.xml    (revision 775934)
>> +++ depends-common.xml    (working copy)
>> @@ -25,7 +25,8 @@
>>    where possible.
>>    </description>
>>
>> -    <loadproperties srcfile="${basedir}/make/depends.properties">
>> +    <copy file="${basedir}/make/depends.properties"
>> tofile="${basedir}/make/depends.properties.ascii"
>> outputencoding="ISO-8859-1" />
>> +    <loadproperties srcfile="${basedir}/make/depends.properties.ascii"
>> encoding="ISO-8859-1">
>>      <filterchain>
>>        <filterreader classname="org.apache.tools.ant.filters.ReplaceTokens">
>>            <param type="token" name="${hy.arch}" value=""/>
>> @@ -34,7 +35,8 @@
>>            <param type="token" name="${hy.platform.variant}" value=""/>
>>        </filterreader>
>>      </filterchain>
>> -    </loadproperties>
>> +    </loadproperties>
>> +    <delete file="${basedir}/make/depends.properties.ascii" />
>>
>>    <macrodef name="download-one-file">
>>        <attribute name="src" />
>>
>> --
>> Oliver Deakin
>> Unless stated otherwise above:
>> IBM United Kingdom Limited - Registered in England and Wales with number
>> 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire
>> PO6 3AU
>>
>>
>>     
>
>   

-- 
Oliver Deakin
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU