Return-Path: Delivered-To: apmail-perl-embperl-archive@www.apache.org Received: (qmail 21302 invoked from network); 22 Dec 2004 11:53:22 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (209.237.227.199) by minotaur-2.apache.org with SMTP; 22 Dec 2004 11:53:22 -0000 Received: (qmail 72902 invoked by uid 500); 22 Dec 2004 11:53:21 -0000 Delivered-To: apmail-perl-embperl-archive@perl.apache.org Received: (qmail 72888 invoked by uid 500); 22 Dec 2004 11:53:20 -0000 Mailing-List: contact embperl-help@perl.apache.org; run by ezmlm Precedence: bulk list-help: list-unsubscribe: list-post: Delivered-To: mailing list embperl@perl.apache.org Received: (qmail 72867 invoked by uid 99); 22 Dec 2004 11:53:20 -0000 X-ASF-Spam-Status: No, hits=0.6 required=10.0 tests=FORGED_RCVD_HELO,HTML_20_30,HTML_MESSAGE,HTML_TITLE_EMPTY X-Spam-Check-By: apache.org Received-SPF: pass (hermes.apache.org: local policy) Received: from rumcajz.pcv.cz (HELO ns.sofist.cz) (81.0.228.129) by apache.org (qpsmtpd/0.28) with SMTP; Wed, 22 Dec 2004 03:53:16 -0800 Received: (qmail 8394 invoked from network); 22 Dec 2004 11:53:47 -0000 Received: from localhost (127.0.0.1) by localhost with SMTP; 22 Dec 2004 11:53:47 -0000 Received: from ns.sofist.cz ([127.0.0.1]) by localhost (lfs [127.0.0.1]) (amavisd-new, port 10024) with SMTP id 07169-02 for ; Wed, 22 Dec 2004 12:53:38 +0100 (CET) Received: (qmail 8303 invoked from network); 22 Dec 2004 11:53:38 -0000 Received: from r3c210.chello.upc.cz (HELO ?192.168.1.2?) (213.220.194.210) by rumcajz.pcv.cz with SMTP; 22 Dec 2004 11:53:38 -0000 Message-ID: <41C96022.2020908@robert.cz> Date: Wed, 22 Dec 2004 12:53:06 +0100 From: RobertCZ User-Agent: Mozilla Thunderbird 1.0 (Windows/20041206) X-Accept-Language: en-us, en MIME-Version: 1.0 To: embperl@perl.apache.org CC: Gerald Richter Subject: Re: Multilingual support / was Re: Urgent: MsgIdExtract problem References: <20041222112628.8D086158DCE@lnx1.i.ecos.de> In-Reply-To: <20041222112628.8D086158DCE@lnx1.i.ecos.de> Content-Type: multipart/alternative; boundary="------------060502020704060908090303" X-Virus-Scanned: by amavisd-new at pcv.cz X-Virus-Checked: Checked X-Spam-Rating: minotaur-2.apache.org 1.6.2 0/1000/N --------------060502020704060908090303 Content-Type: text/plain; charset=ISO-8859-2; format=flowed Content-Transfer-Encoding: 7bit Gerald Richter wrote: >Thanks for your feedback. I try to bring the things into the 2.0 release. > > >>- MsgIdExtract ignores [- $epreq->gettext(expr) -] I don't >>know how to modify the syntax, but it's clearly a bug >> >> > >MsgIdExtract at the moment only knows about [= =], maybe I can try to parse >it out gettext as well, but it will only work for literal strings. I will >think about a solution. > > I believe literal strings would be enough, I was planning something like sprintf $epreq->('%d records founds'), $found; which would then generate "Nalezeno 32 zaznamu" or "32 records found" etc... so the gettext'ed string itself is just a literal string PS What about that additional quotes after running extract and using deafult message if translated string is empty/undef instaed of missing key only? PS2 I'm already using it in my two big projects and it's really REALLY nice ;-) >>-----Original Message----- >>From: RobertCZ [mailto:robert@robert.cz] >>Sent: Wednesday, December 15, 2004 12:49 PM >>To: embperl@perl.apache.org >>Cc: Gerald Richter >>Subject: Multilingual support / was Re: Urgent: MsgIdExtract problem >> >>Gerald Richter wrote: >> >> >>'$Embperl::Syntax::MsgIdExtract::Ids{scalar(%#0%)} = q{} if (!exists >> ($Embperl::Syntax::MsgIdExtract::Ids{scalar(%#0%)})) ;', >> >> >> >> Try to change %#0% to %#\'0% >> >> >> >>OK, now I got it working and it's really cool (with the >>exception of the last couple of bugs). Here are my comments >>(relevant for 2.0RC2): >> >> >>- I changed MsgIdExtract syntax from %#0% to %#\'0%, now it >>correctly extracts more complex expressions, but it adds >>superflous quotes and a space to the extracted expressions, >>it looks like this '\'Test \'' => '', This can be hacked in >>the extract script with something like >> >>foreach my $fn (@files) { >>... >> Embperl::Execute ({use_env => 1, use_redirect_env => 1, >>syntax => 'MsgIdExtract', inputfile => $fn, output => \$out, >>errors => \@errors}) ; ... >>} >>my @correct_keys = map { /^\'\s*(.*?)\s*\'$/ } keys >>%Embperl::Syntax::MsgIdExtract::Ids; >> >>but I think it's a bug >> >> >>- MsgIdExtract ignores [- $epreq->gettext(expr) -] I don't >>know how to modify the syntax, but it's clearly a bug >> >> >>- when messages are dumped with $Data::Dumper::Useqq = 1 it >>break 8bit chars (dump as octal, but translators would go mad >>should they see it) >> >> >>- I find it more practical to dump sorted messages with >>something like $Data::Dumper::Sortkeys = \&{ sub {[ sort { >>$a cmp $b } keys %{$_[0]} ]} } >> >> >>- app file must delete empty (missing) translations to force >>EP use message in the default language, this is quite >>impractical because translators need it in the dictionary >>file and so during the development I need to modify the >>dictionary either for EP or for translators. Now I delete it >>in app file but it's stupid performance-wise >> >>- Robert >> >>------------------------------------- >> >>Just in case somebody needs it, below are necessary files >> >>_app.eo >> >>------------------------------------- >> >>@ISA = ('Embperl::App') ; >> >>sub init { >> my $self = shift ; >> my $r = $self->curr_req ; >> my $fdat = $r->thread->form_hash; >> my $lang = $fdat->{lang} || 'en' ; >> my %messages = %{ do '/home/www/germic/etc/dict/dict.dump' }; >> >> foreach my $l (keys %messages) { >> foreach my $t (keys %{ $messages{$l} }) { >> delete $messages{$l}{$t} unless $messages{$l}{$t}; >> } >> } >> >>#use Data::Dumper; print STDERR "\n\n", Dumper(\%messages), "\n\n"; >> >> push @{$r->messages}, $messages{$lang} ; >> push @{$r->default_messages}, $messages{'en'} if >>($lang ne 'en') ; >> >> return 0; >> >>} >> >>1 ; >> >>------------------------------------- >> >>extract >> >>------------------------------------- >> >>BEGIN >> { >> %Embperl::initparam = (use_env => 1, use_redirect_env => 1) ; >> $ENV{EMBPERL_SESSION_HANDLER_CLASS} = 'no' ; >> } >> >>use Embperl; >>use Data::Dumper ; >> >>use strict; >>use vars qw{$srcpath $dictpath @files @languages $msgids >>@correct_keys}; >> >>$srcpath = '/home/www/germic'; >>$dictpath = '/home/www/germic/etc/dict'; >>@files = split ' ', `find $srcpath -name *.html >>-printf "%p " -or -name *.eo -printf "%p "`; >>@languages = qw(cz de); # en not listed, it's default! >> >>$msgids = do "$dictpath/dict.dump" ; >>die $@ if ($@) ; >>#print Dumper($msgids); >> >>foreach my $fn (@files) { >> my ( $out, @errors ); >> Embperl::Execute ({use_env => 1, use_redirect_env => 1, >>syntax => 'MsgIdExtract', inputfile => $fn, output => \$out, >>errors => \@errors}) ; >> if (@errors) { >> print join ("\n", @errors) ; >> last ; >> } >>} >>#print Dumper(\%Embperl::Syntax::MsgIdExtract::Ids); >> >>@correct_keys = map { /^\'\s*(.*?)\s*\'$/ } keys >>%Embperl::Syntax::MsgIdExtract::Ids; >>#print Dumper(\@correct_keys); >> >>foreach my $lang (@languages) { >> foreach my $k (@correct_keys) { >> $msgids -> {en}{$k} = $k ; >> $msgids -> {$lang}{$k} = '' if (!exists $msgids -> >>{$lang}{$k}) ; >> } >>} >>#print Dumper($msgids); >> >>rename "$dictpath/dict.dump", "$dictpath/dict.bak" ; open FH, >>">$dictpath/dict.dump" or die "Cannot open >>$dictpath/dict.dump ($!)" ; # $Data::Dumper::Indent = 1 ; # >>default is nicer # $Data::Dumper::Useqq = 1 ; # breaks 8bit >>chars $Data::Dumper::Sortkeys = \&{ sub {[ sort { $a cmp $b >>} keys %{$_[0]} ]} } ; # more practical print FH >>Data::Dumper -> Dump ([$msgids], ['msgids']) ; close FH ; >> >>------------------------------------- >> >>test >> >>------------------------------------- >> >>

trans test

>> >>

Test: [= Test =] >>

Hi!: [= Hi! =] >>

Thank You!: [= Thank You! =] >>

Athos, Porthos, Aramis & d'Artagnan: [= Athos, >>Porthos, Aramis & d'Artagnan =]

How's life?: [+ >>$epreq->gettext(qq{How's life?}) +] >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> > > > > --------------060502020704060908090303 Content-Type: text/html; charset=ISO-8859-2 Content-Transfer-Encoding: 7bit Gerald Richter wrote: 

Thanks for your feedback. I try to bring the things into the 2.0 release.
  
- MsgIdExtract ignores [- $epreq->gettext(expr) -] I don't 
know how to modify the syntax, but it's clearly a bug
    

MsgIdExtract at the moment only knows about [= =], maybe I can try to parse
it out gettext as well, but it will only work for literal strings. I will
think about a solution.

I believe literal strings would be enough, I was planning something like

sprintf $epreq->('%d records founds'), $found;

which would then generate "Nalezeno 32 zaznamu" or "32 records found" etc... so the gettext'ed string itself is just a literal string

PS What about that additional quotes after running extract and using deafult message if translated string is empty/undef instaed of missing key only?

PS2 I'm already using it in my two big projects and it's really REALLY nice ;-)




-----Original Message-----
From: RobertCZ [mailto:robert@robert.cz] 
Sent: Wednesday, December 15, 2004 12:49 PM
To: embperl@perl.apache.org
Cc: Gerald Richter
Subject: Multilingual support / was Re: Urgent: MsgIdExtract problem

Gerald Richter wrote: 

		
'$Embperl::Syntax::MsgIdExtract::Ids{scalar(%#0%)} = q{} if (!exists
		($Embperl::Syntax::MsgIdExtract::Ids{scalar(%#0%)})) ;',
		    

	
	Try to change %#0% to %#\'0%
	  


OK, now I got it working and it's really cool (with the 
exception of the last couple of bugs). Here are my comments 
(relevant for 2.0RC2):


- I changed MsgIdExtract syntax from %#0% to %#\'0%, now it 
correctly extracts more complex expressions, but it adds 
superflous quotes and a space to the extracted expressions, 
it looks like this  '\'Test \'' => '', This can be hacked in 
the extract script with something like

foreach my $fn (@files) {
...
    Embperl::Execute ({use_env => 1, use_redirect_env => 1, 
syntax => 'MsgIdExtract', inputfile => $fn, output => \$out, 
errors => \@errors}) ; ...
}
my @correct_keys = map { /^\'\s*(.*?)\s*\'$/ } keys 
%Embperl::Syntax::MsgIdExtract::Ids;

but I think it's a bug


- MsgIdExtract ignores [- $epreq->gettext(expr) -] I don't 
know how to modify the syntax, but it's clearly a bug


- when messages are dumped with $Data::Dumper::Useqq  = 1 it 
break 8bit chars (dump as octal, but translators would go mad 
should they see it)


- I find it more practical to dump sorted messages with 
something like $Data::Dumper::Sortkeys  = \&{ sub {[ sort { 
$a cmp $b } keys %{$_[0]} ]} } 


- app file must delete empty (missing) translations to force 
EP use message in the default language, this is quite 
impractical because translators need it in the dictionary 
file and so during the development I need to modify the 
dictionary either for EP or for translators. Now I delete it 
in app file but it's stupid performance-wise

- Robert

-------------------------------------

Just in case somebody needs it, below are necessary files

_app.eo

-------------------------------------

@ISA = ('Embperl::App') ;

sub init {
        my $self = shift ;
        my $r = $self->curr_req ;
    my $fdat = $r->thread->form_hash;        
        my $lang = $fdat->{lang} || 'en' ;
    my %messages = %{ do '/home/www/germic/etc/dict/dict.dump' };

    foreach my $l (keys %messages) {
        foreach my $t (keys %{ $messages{$l} }) {
            delete $messages{$l}{$t} unless $messages{$l}{$t};
        }
    }

#use Data::Dumper; print STDERR "\n\n", Dumper(\%messages), "\n\n";
        
        push @{$r->messages}, $messages{$lang} ;
        push @{$r->default_messages}, $messages{'en'} if 
($lang ne 'en') ;

    return 0;

}
 
1 ;

-------------------------------------

extract

-------------------------------------

BEGIN 
    {
    %Embperl::initparam = (use_env => 1, use_redirect_env => 1) ;
    $ENV{EMBPERL_SESSION_HANDLER_CLASS} = 'no' ;
    }    
    
use Embperl;
use Data::Dumper ;

use strict;
use vars qw{$srcpath $dictpath @files @languages $msgids 
@correct_keys};

$srcpath    = '/home/www/germic';
$dictpath    = '/home/www/germic/etc/dict';
@files        = split ' ', `find $srcpath -name *.html 
-printf "%p " -or -name *.eo -printf "%p "`;
@languages    = qw(cz de); # en not listed, it's default!

$msgids = do "$dictpath/dict.dump" ;
die $@ if ($@) ;
#print Dumper($msgids);

foreach my $fn (@files) {
    my ( $out, @errors );
    Embperl::Execute ({use_env => 1, use_redirect_env => 1, 
syntax => 'MsgIdExtract', inputfile => $fn, output => \$out, 
errors => \@errors}) ;
    if (@errors) {
        print join ("\n", @errors) ;
        last ;
    }
}
#print Dumper(\%Embperl::Syntax::MsgIdExtract::Ids);

@correct_keys = map { /^\'\s*(.*?)\s*\'$/ } keys 
%Embperl::Syntax::MsgIdExtract::Ids;
#print Dumper(\@correct_keys);

foreach my $lang (@languages) {
    foreach my $k (@correct_keys) {
        $msgids -> {en}{$k} = $k ;
        $msgids -> {$lang}{$k} = '' if (!exists $msgids -> 
{$lang}{$k}) ;
    }
}
#print Dumper($msgids);

rename "$dictpath/dict.dump", "$dictpath/dict.bak" ; open FH, 
">$dictpath/dict.dump" or die "Cannot open 
$dictpath/dict.dump ($!)" ; # $Data::Dumper::Indent = 1 ; # 
default is nicer # $Data::Dumper::Useqq  = 1 ; # breaks 8bit 
chars $Data::Dumper::Sortkeys  = \&{ sub {[ sort { $a cmp $b 
} keys %{$_[0]} ]} }  ; # more practical print FH 
Data::Dumper -> Dump  ([$msgids], ['msgids']) ; close FH ;

-------------------------------------

test

-------------------------------------

<h1>trans test</h1>

<p>Test: <b>[= Test =]</b>
<p>Hi!: <b>[= Hi! =]</b>
<p>Thank You!: <b>[= Thank You! =]</b>
<p>Athos, Porthos, Aramis & d'Artagnan: <b>[=  Athos, 
Porthos, Aramis & d'Artagnan =]</b> <p>How's life?: <b>[+ 
$epreq->gettext(qq{How's life?}) +]<b>

--------------060502020704060908090303--