cordova-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrey Kurdumov <kant2...@googlemail.com>
Subject Re: Crowdin and translation process
Date Wed, 04 Feb 2015 11:07:32 GMT
I made script which will ensure that translated MD files from Crowdin will
have correct license headers.

Please review:
https://github.com/apache/cordova-docs/pull/264

Usage:
from cordova-docs repository root directory invoke
./bin/fixyaml

Currently this script only working over cordova-docs repository, not
verifying plugin repositories.
If I download latest versions seems, that only one issue with license
headers we have:

docs/<lang>/guide/next/index.md has missing license header.
That's because to Crowdin was uploaded file without license header.


2015-02-04 10:11 GMT+06:00 Victor Sosa <sosah.victor@gmail.com>:

> Hello community.
>
> So that everybody is aware about the proceedings on this topic.
>
> Andrey and I just had a quick call about this topic and here the agreement
>
> We want to still continue using Crowdin as our tool for automatic
> translation, but we do acknowledge the problems depicted on this thread
> (autolinking and formatting). In order to fix these problems, Andrey worked
> on a couple of validation scripts that we can integrate in the translation
> process, but for this to happen we need to do the following (Andrey, please
> keep me honest):
>
> 1. I'll reach out the Crowdin folks to see whether they have any API to
> put translated documents back in CrowdIn. This way we can run the
> validation scripts locally and then put the fixed docs back and run the
> usual Crowdin build. If no API is available then try to find a workaround
> to this problem (as of now, we couldn't find a way to do it).
>
> 2. As a low priority task for now, I'll refactor the current translation
> script (shell script) to something more cross-platform (ideas about the
> best solution/technology are welcome)
>
> 3. Andrey will continue his work making sure the output of the validation
> scripts is as expected.
>
> Hope I didn't miss anything from our chat.
>  On Feb 3, 2015 8:32 PM, "Victor Sosa" <sosah.victor@gmail.com> wrote:
>
>> Hi Andrey
>>
>> Yes, in about an hour as agreed
>> On Feb 3, 2015 8:22 PM, "Andrey Kurdumov" <kant2002@googlemail.com>
>> wrote:
>>
>>> Hi,
>>>
>>> will you able chat today 9:30 PM CST as we plan earlier?
>>>
>>> 2015-01-30 8:47 GMT+06:00 Victor Sosa <sosah.victor@gmail.com>:
>>>
>>>> Cool! So I'll book it on my calendar, Tuesday 9:30pm. That's Your
>>>> Wednesday 9:30am. Talk to you then.
>>>>
>>>> Have a great weekend!
>>>> On Jan 29, 2015 8:39 PM, "Andrey Kurdumov" <kant2002@googlemail.com>
>>>> wrote:
>>>>
>>>>> I'm fine for that time.
>>>>>
>>>>> 2015-01-30 8:37 GMT+06:00 Victor Sosa <sosah.victor@gmail.com>:
>>>>>
>>>>>> So let's schedule a hangout next Wednesday in the morning, which
is
>>>>>> my Tuesday night. If I recall correctly we are 12 hours apart from
each
>>>>>> other (I'm in GMT -6). How about 9:30 AM for you?
>>>>>> On Jan 29, 2015 7:52 PM, "Andrey Kurdumov" <kant2002@googlemail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> If this is working for you, we could chat.
>>>>>>> Morning is start for me, hopefully it is not too late for you.
I
>>>>>>> could chat in next 4 hours or more without problem.
>>>>>>>
>>>>>>> 2015-01-30 6:27 GMT+06:00 Victor Sosa <sosah.victor@gmail.com>:
>>>>>>>
>>>>>>>> Hi Andrey. Thanks for the reminder.
>>>>>>>>
>>>>>>>> In the process in step 4 you are correct. This "manual" step
is
>>>>>>>> actually going to CrowdIn and launch the build from there.
>>>>>>>>
>>>>>>>> I agree that after 5, we need to establish some kind of mechanism
>>>>>>>> to validate the two scenarios you state here (formatting
 errors and
>>>>>>>> autolinking). I presume this translationreport script is
the one you
>>>>>>>> created and will be useful to validate the autolinking problems.
And as you
>>>>>>>> state, it is quite difficult to maintain each language and
therefore we
>>>>>>>> have to live with the problems we have for the time being
:S
>>>>>>>>
>>>>>>>> Let me tell you this, why don't you and schedule a hangout
to speed
>>>>>>>> up this conversation and convey an agreement? I'm fine if
you want to book
>>>>>>>> the hangout in your morning, which is my night ;-), just
let me know your
>>>>>>>> availability (we can have a quick chat on gmail privately
to not spam the
>>>>>>>> list)
>>>>>>>>
>>>>>>>> 2015-01-29 9:08 GMT-06:00 Andrey Kurdumov <kant2002@googlemail.com>
>>>>>>>> :
>>>>>>>>
>>>>>>>> Hi, want to bump this thread. Do you have any thoughts how
I could
>>>>>>>>> fix current documentation issues without interfering
with you?
>>>>>>>>>
>>>>>>>>> 2015-01-23 13:46 GMT+06:00 Andrey Kurdumov <
>>>>>>>>> kant2002@googlemail.com>:
>>>>>>>>>
>>>>>>>>>> Thanks for explanation. The process is more like
i think.
>>>>>>>>>> One thing about which I'm not sure is #4. I assume
that this is
>>>>>>>>>> some step in CrowdIn which generated ZIP file from
the latest translations.
>>>>>>>>>> Correct me if I wrong.
>>>>>>>>>>
>>>>>>>>>> a) My feeling how I would like to work:
>>>>>>>>>> 1. The script downloads the latest updates to your
local Got repo.
>>>>>>>>>> 2. Adds the docs in English to CrowdIn.
>>>>>>>>>> 3. Manually start the translation via a machine in
CrowdIn
>>>>>>>>>> 4. Once translated, manually run a build of the docs.
>>>>>>>>>> 5. Using the script, download the changes in documentation
to
>>>>>>>>>> your local repo.
>>>>>>>>>>
>>>>>>>>>> X. Run validation scripts.
>>>>>>>>>> X.a. In case of any translation error (autolinking
for example)
>>>>>>>>>> go to CrowdIn and start from step #3.
>>>>>>>>>> X.b In case of some minor formatting errors, correct
them
>>>>>>>>>> manually and continue to step #6.
>>>>>>>>>>
>>>>>>>>>> 6. Commit changes to Git.
>>>>>>>>>>
>>>>>>>>>> I just want to add validation step which will not
allow to commit
>>>>>>>>>> specifically for Russian language, since this is
only language which I
>>>>>>>>>> could support.
>>>>>>>>>> Other languages I don't plan to support, and from
my perspective
>>>>>>>>>> they are broken.
>>>>>>>>>>
>>>>>>>>>> Here the small Node.js utility which could help you
test broken
>>>>>>>>>> auto-linking on any language you care.
>>>>>>>>>> Run
>>>>>>>>>>  ./bin/translationreport ru edge -v
>>>>>>>>>> from the root folder of [cordova-docs] repository.
>>>>>>>>>>
>>>>>>>>>> I will never go to step #6 until I fix every errors
which I found
>>>>>>>>>> on step X. This is how I feel it should be. But I
open for alternatives
>>>>>>>>>> which allows me improve quality of Russian translation,
>>>>>>>>>>
>>>>>>>>>> Regarding make validation step possible in CrowdIn.
So far, I
>>>>>>>>>> don't see any method which allows verify of translation
without downloading
>>>>>>>>>> it first from CrowdIn (see https://crowdin.com/page/api).
>>>>>>>>>>
>>>>>>>>>> I really try make good translation using just CrowdIn,
but don't
>>>>>>>>>> see any opportunity to fix broken MD files inside
CrowdIn.
>>>>>>>>>> If you have any ideas how I could fix errors in the
links I show
>>>>>>>>>> you using just CrowdIn, I would be all-in for that.
>>>>>>>>>>
>>>>>>>>>> Best regards,
>>>>>>>>>> Andrey
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> 2015-01-22 8:56 GMT+06:00 Victor Sosa <sosah.victor@gmail.com>:
>>>>>>>>>>
>>>>>>>>>>> Hi Andrey, sorry my late replay. Here my 2 cents.
>>>>>>>>>>>
>>>>>>>>>>> This is what the current translation process
does:
>>>>>>>>>>>
>>>>>>>>>>> 1. The script downloads the latest updates to
your local Got
>>>>>>>>>>> repo.
>>>>>>>>>>> 2. Adds the docs in English to CrowdIn.
>>>>>>>>>>> 3. Manually start the translation via a machine
in CrowdIn
>>>>>>>>>>> 4. Once translated, manually run a build of the
docs.
>>>>>>>>>>> 5. Using the script, download the changes in
documentation to
>>>>>>>>>>> your local repo.
>>>>>>>>>>> 6. Commit changes to Git.
>>>>>>>>>>>
>>>>>>>>>>> If a translator modifies a doc at some point
in CrowdIn, I'll
>>>>>>>>>>> take the changes in the next build.
>>>>>>>>>>>
>>>>>>>>>>> As Lisa stated, any changes made in the translated
docs outside
>>>>>>>>>>> of CrowdIn will end up ignored in the next build.
>>>>>>>>>>>
>>>>>>>>>>> If I understood correctly, you propose to validate
the generated
>>>>>>>>>>> docs using the tool you created. I wonder if
we can add the validation
>>>>>>>>>>> process to modify the docs in CrowdIn instead
of directly doing it in Git.
>>>>>>>>>>>
>>>>>>>>>>> Correct me if wrong, but I think this is the
way to go in order
>>>>>>>>>>> not just to keep the current process but to enhance
it. Let me know what
>>>>>>>>>>> you think.
>>>>>>>>>>>
>>>>>>>>>>> P.S. I'm in Central Timezone, which means I'm
12 hours away from
>>>>>>>>>>> you. Communication is an exciting topic here
:-D
>>>>>>>>>>> On Jan 20, 2015 9:24 PM, "Andrey Kurdumov" <
>>>>>>>>>>> kant2002@googlemail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> My main concern is that current translation
not good.
>>>>>>>>>>>> Proof: http://cordova.apache.org/docs/es/edge/index.html
or
>>>>>>>>>>>> http://cordova.apache.org/docs/ru/edge/index.html
>>>>>>>>>>>>
>>>>>>>>>>>> These links contain two major issues with
translated docs.
>>>>>>>>>>>> a) License headers
>>>>>>>>>>>> b) Missing links which could lead to missing
parts of docs.
>>>>>>>>>>>> other issues are also present which also
introduced by our
>>>>>>>>>>>> tools.
>>>>>>>>>>>>
>>>>>>>>>>>> ----------------------------------------------------
>>>>>>>>>>>> First kind of errors was introduced as of
Cordova 3.4 or like
>>>>>>>>>>>> that.
>>>>>>>>>>>> There no way fix that using CrowdIn, so I
fix that in the Git
>>>>>>>>>>>> repo byt sending pull request
>>>>>>>>>>>> https://github.com/apache/cordova-docs/pull/250
>>>>>>>>>>>>
>>>>>>>>>>>> Example of changes
>>>>>>>>>>>>
>>>>>>>>>>>> https://github.com/apache/cordova-docs/commit/691f0f016ef4932e9cd224f49bb9641161b726e1
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> ----------------------------------------------------
>>>>>>>>>>>> Second kind of errors (missing links) could
be verified by
>>>>>>>>>>>> running new tool
>>>>>>>>>>>>
>>>>>>>>>>>>  ./bin/translationreport ru edge -v
>>>>>>>>>>>> or
>>>>>>>>>>>>  ./bin/translationreport ru edge
>>>>>>>>>>>> for more details.
>>>>>>>>>>>>
>>>>>>>>>>>> ----------------------------------------------------
>>>>>>>>>>>>
>>>>>>>>>>>> Really I don't trust output of CrowdIn due
to way we generate
>>>>>>>>>>>> documentation. It is very easy translate
sentence and broke our
>>>>>>>>>>>> autolinking in the documentation. Current
process is not enough
>>>>>>>>>>>> for me, since before commit changes to Git
for Russian translation
>>>>>>>>>>>> I want to manually generate Russian documentation
and verify
>>>>>>>>>>>> quality of produced output using automated-tools
and by manual
>>>>>>>>>>>> spot-checking.
>>>>>>>>>>>> If any of tools report about problem with
translation I would
>>>>>>>>>>>> not commit any changes until I resolve issue.
If that mean that
>>>>>>>>>>>> I have to manually insert blank lines after
pulling changes
>>>>>>>>>>>> from CrowdIn, I will do that.
>>>>>>>>>>>>
>>>>>>>>>>>> I have doubts that this is practical follow
this approach for
>>>>>>>>>>>> other languages, that's why separation. If
you are ready to
>>>>>>>>>>>> go with that approach for all other languages,
I would be more
>>>>>>>>>>>> then happy not to separate Russian. I want
find solution/process where all
>>>>>>>>>>>> we would work with same set of tools, plus
I will use
>>>>>>>>>>>> additional tools to raise quality bar for
Russian.
>>>>>>>>>>>>
>>>>>>>>>>>> Want to notice that I don't want to dump
CrowdIn from the
>>>>>>>>>>>> process, it is very good to for translation,
>>>>>>>>>>>> but not for the verification of the translated
results. Reason
>>>>>>>>>>>> that I made pull request on GitHub directly,
>>>>>>>>>>>> it is because CrowdIn does not allow do what
I need.
>>>>>>>>>>>>
>>>>>>>>>>>> Hopefully this is clears picture from my
side.
>>>>>>>>>>>>
>>>>>>>>>>>> BTW: another problem is that I GMT +6 which
is 11 hours
>>>>>>>>>>>> different from East side of  US and 14 from
West side of US.
>>>>>>>>>>>> Add some communication problems.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> 2015-01-20 22:43 GMT+06:00 Lisa Seacat DeLuca
<
>>>>>>>>>>>> ldeluca@us.ibm.com>:
>>>>>>>>>>>>
>>>>>>>>>>>>> Andrey~
>>>>>>>>>>>>>
>>>>>>>>>>>>> I'm a little confused by your request.
 Are you saying that
>>>>>>>>>>>>> you make changes to the translation outside
of the Crowdin tool?  If that's
>>>>>>>>>>>>> the case, yes, all of your changes are
going to be lost every time.  You'll
>>>>>>>>>>>>> need to make the changes within Crowdin.
 When the script is ran to make
>>>>>>>>>>>>> sure the latest documentation is pushed
to the Crowdin tool it doesn't
>>>>>>>>>>>>> override any translations that have already
been done.  It just adds
>>>>>>>>>>>>> updates to the documentation (new sentences,
new code, etc) or changes that
>>>>>>>>>>>>> were made.  So unless sections of the
english version of the documentation
>>>>>>>>>>>>> has been changes your Russian updates
shouldn't be affected.  I see no
>>>>>>>>>>>>> reason why you couldn't own the Russian
part of the translation process.
>>>>>>>>>>>>> Whenever Victor goes to pull down the
translations if the Russian
>>>>>>>>>>>>> translations aren't at 100% he can talk
to you.  Does that sound
>>>>>>>>>>>>> reasonable?  Did I understand your concern
correctly?
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Lisa
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> [image: Inactive hide details for Andrey
Kurdumov
>>>>>>>>>>>>> ---01/20/2015 01:22:53 AM---Hi, I start
working on the improving Cordova
>>>>>>>>>>>>> documentatio]Andrey Kurdumov ---01/20/2015
01:22:53 AM---Hi,
>>>>>>>>>>>>> I start working on the improving Cordova
documentation for Russian language.
>>>>>>>>>>>>>
>>>>>>>>>>>>> From: Andrey Kurdumov <kant2002@googlemail.com>
>>>>>>>>>>>>> To: dev@cordova.apache.org, Victor Adrian
Sosa Herrera <
>>>>>>>>>>>>> sosah.victor@gmail.com>
>>>>>>>>>>>>> Date: 01/20/2015 01:22 AM
>>>>>>>>>>>>> Subject: Crowdin and translation process
>>>>>>>>>>>>> ------------------------------
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>
>>>>>>>>>>>>> I start working on the improving Cordova
documentation for
>>>>>>>>>>>>> Russian language.
>>>>>>>>>>>>> Earlier I put fix for the YAML license
headers directly to
>>>>>>>>>>>>> Github for all
>>>>>>>>>>>>> translations.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Right now after I put documentation translation
checker tool,
>>>>>>>>>>>>> I see issues
>>>>>>>>>>>>> which I would like to improve Russian
translation.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I download latest translation from Crowdin
and compare that
>>>>>>>>>>>>> field with docs
>>>>>>>>>>>>> in the source control. What I found is
that YAML fixes is not
>>>>>>>>>>>>> in CrowdIn.
>>>>>>>>>>>>>
>>>>>>>>>>>>> 1. This raises question how we could
could work on translation
>>>>>>>>>>>>> with Victor
>>>>>>>>>>>>> together effectively, not impacting each
other?
>>>>>>>>>>>>> When documentation files is uploaded
to the Crowdin?
>>>>>>>>>>>>> When translation is collected from Crowdin?
>>>>>>>>>>>>> Are there any rules which could be used,
so I could prepare to
>>>>>>>>>>>>> these cycles?
>>>>>>>>>>>>>
>>>>>>>>>>>>> 2. Current script just download everything
and commit it in the
>>>>>>>>>>>>> cordova-docs repository.
>>>>>>>>>>>>> What I think is that it could be effective
is that I take
>>>>>>>>>>>>> responsibility
>>>>>>>>>>>>> for maintaining Russian translation and
Victor would maintain
>>>>>>>>>>>>> other
>>>>>>>>>>>>> translations, like it was before. And
downloading script could
>>>>>>>>>>>>> be modified
>>>>>>>>>>>>> to include language which should be downloaded,
so this allow
>>>>>>>>>>>>> explicit
>>>>>>>>>>>>> control which languages is created. I
recently cleanup a lot
>>>>>>>>>>>>> of files from
>>>>>>>>>>>>> probably Crowdin mistakes, and don't
want that this would be
>>>>>>>>>>>>> introduced
>>>>>>>>>>>>> again.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Victor Adrian Sosa Herrera
>>>>>>>> IBM Software Engineer
>>>>>>>> Guadalajara, Jalisco
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>
>>>

Mime
  • Unnamed multipart/related (inline, None, 0 bytes)
View raw message