Mailing-List: contact users-help@subversion.apache.org; run by ezmlm
Precedence: bulk
Received-SPF: pass (nike.apache.org: domain of gcvsu-users@m.gmane.org
 designates 80.91.229.3 as permitted sender)
To: users@subversion.apache.org
From: =?utf-8?b?0JLQsNGA0YTQvtC70L7QvNC10LXQsg==?=
 =?utf-8?b?0JjQs9C+0YDRjA==?= <i3v@mail.ru>
Subject: Re: Unicode characters in filenames on windows
Date: Wed, 12 Jun 2013 23:18:36 +0000 (UTC)
Lines: 198
Message-ID: <loom.20130613T010100-569@post.gmane.org>
References: <004e01ce6707$ee704450$cb50ccf0$@mail.ru>
 <51B7DEF8.9080209@wandisco.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
User-Agent: Loom/3.14 (http://gmane.org/)

Branko Čibej <brane <at> wandisco.com> writes:

> 
> On 12.06.2013 02:58, Варфоломеев Игорь wrote:
> > Hi all,
> > I'm still not sure if it's a bug, or if I'm doing something wrong. But
I'm unable to get TortoiseSVN 
> > command-line tool to work with files with UTF-8 characters in their name. 
> > (1.7.10 r1485443, part of TortoiseSVN 1.7.13  <at> Win7 x64) 
> >
> >
> > I've posted the following message here
> > (
http://tortoisesvn.tigris.org/ds/viewMessage.do?dsForumId=4061&dsMessageId=3057388
)
> > but was suggested 
> > (
http://tortoisesvn.tigris.org/ds/viewMessage.do?dsForumId=4061&dsMessageId=3057494
)
> > to re-address it to users <at> subversion.apache.org :
> >
> >
> >
---------------------------------------------------------------------------------
> > *** THE ISSUE ***
> >
> >
> > Workflow:
> >
> > 1. Create "c:\temp\UNCtest\R_UNCtest\" folder
> > 2. Create a repository with default file structure in it
> > 3. Checkout "trunk" dir to "c:\temp\WC\trunk"
> > 4. Create file "c:\temp\WC\trunk\1‐2.txt" ,
> > note, that filename consists of 3 symbols, and the one in the middle is
"HYPHEN" or ‐, (see
> http://www.fileformat.info/info/unicode/char/2010/index.htm )
> >
> > 5. Add and commit this file with Tortoise GUI.
> > (this works OK)
> > 6. start windows cmd
> > 7. make sure your cmd is set to use UTF-8 compatible font, for example,
"Consolas" (see
>
http://stackoverflow.com/questions/10764920/utf-16-on-cmd-exe/10765469#10765469
).
> > 8. navigate to "c:\temp\WC\trunk"
> > 9. type "dir" - you should see the listing correctly, including
"1‐2.txt" file
> > 10. Type "mkdir 1‐2" - this should correctly create a directory.
> > 11. Type "svn info 1‐2.txt"
> > Result:
> > --------------------------------------------------
> > svn: warning: W155010: The node 'C:\TEMP\UNCtest\WC\trunk\1?2.txt' was
not found
> > .
> >
> > svn: E200009: Could not display info for all targets because some
targets don't
> > exist
> 
> I believe this happens because the "chcp" command changes the OEM
> (console) code-page, but Subversion uses the ANSI (Windows) code page
> for input and output conversion. In other words, the "chcp" does not
> affect the command-line client in any way.
> 
> > * Am I doing something wrong?
> 
> Not as such. :)
> 
> > * Or could this situation be treated as a bug?
> > * Or, maybe a “feature request”?
> 
> I think the only way to actually get this right is to change the way we
> read from and write to the console on Windows. Instead of converting
> strings to some native encoding and using the ordinary output functions,
> we should convert to UTF-16 and use wide-char output functions instead.
> 


I'm still not sure what should I do next...
Should I re-address this to dev@subversion.apache.org ?

Anyway, here's a batch file, reproducing the issue:
https://skydrive.live.com/redir?resid=8FD4EC3161ABA67A!916 

==========================================================================
@echo off

rem * This batch file aims to reproduce an issue with Unicode support in svn
CLI.
rem * This file is in UTF-8 65001 encoding, and it should not be converted
to ANSI or etc.
rem * I highly recommend to change default cmd font to "Consolas", for
Unicode support,
rem   see
http://stackoverflow.com/questions/10764920/utf-16-on-cmd-exe/10765469#10765469
rem * This file is based on template from 
rem  
http://subversion.apache.org/docs/community-guide/issues.html#reporting-bugs
rem 
rem *** TESTED ON ***
rem Win7 x64 - working
rem WinXP x64 - not working (unable to create a file with valid name)
rem 1.7.10 r1485443, part of TortoiseSVN 1.7.13 x64
rem 


:defineCommands
rem You might need to adjust these lines to point to your
rem compiled-from-source Subversion binaries, if using those:
for %%i in (svn.exe) do set SVN="%%~$PATH:i"
for %%i in (svnadmin.exe) do set SVNADMIN="%%~$PATH:i"

:defineUrls
rem Only supported access method: file://. If http:// or svn://, then
rem you'll have to configure it yourself first.
set URL=file:///%CD%/repos
set URL=%URL:\=/%
echo Base url for repo: %URL%

:cleanAllDirsAndCreateRepo
if exist repos rmdir /s /q repos
if exist import-me rmdir /s /q import-me
if exist wc rmdir /s /q wc
%SVNADMIN% create repos

:prepareGreekTree
echo Making a Greek Tree for import...
mkdir import-me
mkdir import-me\trunk
mkdir import-me\tags
mkdir import-me\branches

rem -------------------------------------------------------------
rem HERE WE GO:


chcp 65001

rem Note: Some people say, that "chcp 65001" on a line by itself aborts the
batch file.
rem And this seem to be true for Windows XP x64. Though, it works OK on Win7
x64.
rem see
http://stackoverflow.com/questions/2182568/batch-script-is-not-executed-if-chcp-was-called/2462138#2462138
rem If you're experiencing this issue, you may simply type "chcp 65001" before 
rem launching this script.


echo This is the file '1‐2.txt'.        > import-me\trunk\1‐2.txt


rem Note1: The symbol between "1" and "2" is "HYPHEN" or ‐, 
rem see http://www.fileformat.info/info/unicode/char/2010/index.htm 
rem If you're getting incorrect filename, like "1тАР2.txt" (or something) 
rem instead of "1‐2.txt", make sure you've typed "chcp 65001".
rem
rem Note2: "import-me\trunk\1‐2.txt" itself would have 65001 encoding too.


echo Importing it...
cd import-me
%SVN% import -q -m "Initial import." %URL%
cd ..

:checkoutTrunk
echo Checking out working copy..
%SVN% co -q %URL%/trunk wc


echo Here's the Error1:
%SVN% info wc/1‐2.txt

rem  | svn: warning: W155010: The node 'F:\temp\wc\1?2.txt' was not found.
rem  | 
rem  | svn: E200009: Could not display info for all targets because some
targets don't
rem  | exist


echo This is a modification.        >> wc\1‐2.txt

echo Here's the Error2:
%SVN%  commit  wc\1‐2.txt
rem  |svn: E200009: Commit failed (details follow):
rem  |svn: E200009: 'F:\temp\wc\1?2.txt' is not under version control


rem I've not tested all svn commands.. there may be others...

goto :eof

==========================================================================

Igor