Difference between revisions of "On Boarding a Localyzer Project"

From Lingoport Wiki
Jump to: navigation, search
(Configure the Jenkins job)
(Verifying the Automation Job)
(44 intermediate revisions by 2 users not shown)
Line 22: Line 22:
 
=== Resource File types ===
 
=== Resource File types ===
   
* Files types (for instance .properties, .resx, etc.)
+
* Files types (for instance .properties, .resx, etc.). For more information on Resource file types and what is supported, please see [[Resource Files]]
* Naming convention for the base file: for instance '''<filename>_en.properties''' or '''<filename>.properties'''
+
* Naming convention for the base file: for instance '''<filename>_en.properties''' or '''<filename>.properties'''
  +
* The directory structure for the resource files.
 
* Translated file naming convention: for instance '''<filename>_fr_FR.properties'''
 
* Translated file naming convention: for instance '''<filename>_fr_FR.properties'''
 
* The files or directories which follow the above convention but should NOT be translated. For instance **/config*.properties
 
* The files or directories which follow the above convention but should NOT be translated. For instance **/config*.properties
Line 29: Line 30:
 
=== Localization Vendor ===
 
=== Localization Vendor ===
 
* Who is the translation management system and how are the resource files going to be transferred to them?
 
* Who is the translation management system and how are the resource files going to be transferred to them?
* Lingoport supports Local vendors, FTP and SFTP, and WorldServer and Lingotek and also other vendors.
+
* Lingoport supports multiple vendors including Local vendors, FTP and SFTP, WorldServer, Lingotek, Systran and AWS Amazon Translate.
   
  +
For more information about selecting a L10n Vendor for the project see [[L10n_Vendors_and_Integration#Access_Level|Creating L10n Vendors]]
= On Boarding the Automation Job =
 
  +
  +
= On Boarding the LRM Automation Job =
   
 
== Jenkins ==
 
== Jenkins ==
Line 39: Line 42:
 
[[File:automation_tab_new_item.jpg|700px]]
 
[[File:automation_tab_new_item.jpg|700px]]
   
* Make a copy of <code>Lingoport.SampleLRM</code> (if this project uses LRM)
+
* Make a copy of <code>Lingoport.SampleLRM</code>
 
* Set the name of the new job to '''<groupname>.<projectname>'''
 
* Set the name of the new job to '''<groupname>.<projectname>'''
 
* Make sure '''Add to current view''' is checked and select '''OK'''
 
* Make sure '''Add to current view''' is checked and select '''OK'''
Line 46: Line 49:
   
 
==== General ====
 
==== General ====
  +
[[File:GeneralTab.png|700px]]
   
* '''General''' section: Select the GitHub project and fill in the Project url
+
*'''General''' section: Select the GitHub project and fill in the Project url
* '''Throttle Current Builds''' should be checked.
+
*'''Throttle Current Builds''' should be checked.
* '''LRM Throttle''' should be checked.
+
*'''LRM Throttle''' should be checked.
   
 
==== Source Code Management ====
 
==== Source Code Management ====
 
[[File:source_code_management.jpg|700px]]
 
[[File:source_code_management.jpg|700px]]
  +
* '''Source Code Management''' section: Set up the Repository, Credentials, and Branch
 
  +
'''Source Code Management''' section: Set up the Repository, Credentials, and Branch. For example if the Git repository is Acme/myproject and the code to work with is in the 'development' branch, then set the Repository URL to '''<nowiki>https://github.com/Acme/myproject.git</nowiki>'''. It is often easy to copy this from GitHub. For the branch, make sure to define it as '''*/development'''. If the '*/' is left off, there will be problems.
  +
 
==== Build Triggers ====
 
==== Build Triggers ====
  +
[[File:BuildTriggers.png|700px]]
  +
  +
These are left unchecked by default, but the '''Build Periodically''' can be set to build on a repeating basis.
  +
 
==== Build Environment ====
 
==== Build Environment ====
  +
[[File:BuildEnvironment.png|700px]]
  +
  +
Leave these unchecked.
  +
 
==== Build ====
 
==== Build ====
==== Post Build Actions ====
 
   
  +
[[File:Build1.png|700px]]
You may use <code>Lingoport.SampleLRM</code> as a template to create your project or create a New Item and add "run LRM" in the Build section (see the drop down).
 
   
  +
* The '''Top Level Directory''' is defaulted to look at the whole repository. If the resource files are located under a resources directory, this can be changed to '''<code>$WORKSPACE/resources</code>''' .
In the '''Build''' section:
 
  +
* '''Locales''' are defaulted to common values. Remove and add locales to get the appropriate translation languages. '''Pseudo locale''' defaults to 'eo'.
* '''Run LRM'''
 
** '''Copy Settings''': Use this if you have a project already set up with a similar configuration. This can save configuration time. If there is no other project, just leave it unfilled.
+
* '''Copy Settings''': Use this if you have a project already set up with a similar configuration. This can save configuration time. If there is no other project, just leave it unfilled.
  +
* Check the '''Send Unique File Names''' if you want the prep kit files to have unique file names. The default is not to send unique file names. If you have multiple file names that have the same name you may want to check the box. For more information see [[LRM_Unique_File_Names|LRM Unique File Names]].
** '''General Settings''':
 
*** Set the '''Project Description''' as a comment on the project.
+
* Advanced: '''Detect Errors''': It is unlikely you need to change the default settings.
  +
** '''Missed Translation Error''': (Default: Unchecked) If you check this box, when a string is returned back from translation with the same text as the original one, an error will be raised. Be cautious with this check: Population for instance is the same word in English and in French.
*** Set the '''Top Level Directory''' to the location of the LRM resource files. <code>$WORKSPACE</code> is the default, but lower level directories can be specified.
 
  +
** '''Parameter Mismatch Error''': (Default: Checked) Keep this check if you'd like to make sure that when a parameter(s) is (are) sent in a text, say "Hello {userName}", the same parameter(s) is (are) sent back untranslated. If the French translation is "Bonjour {nom d'utilisateur}", the 'userName' variable cannot be replaced at run time in the application.
*** Set the '''Send Unique File Names''' to 1 if you want the prep kit files to have unique file names. The default is not to send unique file names. If you have multiple file names that have the same name you may want to change the flag to 1. For more information see [[LRM_Unique_File_Names|LRM Unique File Names]]
 
** '''Detect Errors''': It is unlikely you need to change the default settings.
 
*** '''Missed Translation Error''': (Default: Unchecked) If you check this box, when a string is returned back from translation with the same text as the original one, an error will be raised. Be cautious with this check: Population for instance is the same word in English and in French.
 
*** '''Parameter Mismatch Error''': (Default: Checked) Keep this check if you'd like to make sure that when a parameter(s) is (are) sent in a text, say "Hello {userName}", the same parameter(s) is (are) sent back untranslated. If the French translation is "Bonjour {nom d'utilisateur}", the 'userName' variable cannot be replaced at run time in the application.
 
** '''Locales'''
 
*** Locales must be specified using the traditional Java format, for instance <code>fr_FR</code> for France French. The locales must then be applied to each resource file type for this project and the file naming or the directory naming may use another format entirely. For instance, resx files use dots (.) and dashes(-) as in <code>resource.fr-FR.resx</code>. The locales in this section must still be specified with underscores as in Java.
 
*** The '''Default Locale''' is the base language used in the project. It is generally English (en_US or en), but it can be any language.
 
*** The '''Pseudo Locale''' A locale outside the target translations locales used to show the application user-facing strings have been moved outside of the code and into resource file; Unicode characters are handled properly and the layout of the application handles different string lengths gracefully.
 
*** The '''Track Back Locale''' is a locale outside the target translation locales used to show where in the code the string is located. This may be useful when something is off on the screen and the developers need to know where that string is located.
 
*** The '''Target Locales''' are the locales to be translated into. Four common locales are defaulted, but can easily be changed and added to
 
** '''Resource File Extensions''': You can have more than one file extension. For example, you may have .json file and .properties files in the same repository. '''For each file''' extension, fill in or modify the following fields:
 
*** Extension: the resource file extension, for instance properties (without the '.'), json, resx, etc.
 
*** Filename Pattern:
 
**** The <code>*</code> (star) character represents the filename without the extension or the locales;
 
**** <code>l</code> represents the locale's language, for instance 'fr' for French
 
**** <code>c</code> represents the locale's country, for instance 'FR' for France
 
**** <code>v</code> represents the locale's variant. That's less frequent. is the variant.
 
*** A few examples may help:
 
**** <code>*_l_c_v</code>: If a target locale is '''fr_FR''' and '''messages_en.properties''' needs to be translated, the French file will be '''message_fr_FR.properties'''
 
**** <code>*.l-c-v</code>: If a target locale is '''fr_FR''' and '''messages.resx''' needs to be translated, the French file will be '''messages.fr-FR.resx'''
 
**** <code>*-l-c-v</code>: If a target locale is '''fr_FR''' and '''messages-en.json''' needs to be translated, the French file will be '''messages-fr-FR.json'''
 
** '''Directories''': Directories to include and exclude from LRM process
 
** '''Notifications''': Who to notify in various situations. Multiple email addresses can be assigned to each entry.
 
Select the '''Save''' button. '''Enable''' the project and '''Build Now'''. The Jenkins project should run and push the results to the Lingoport Dashboard.
 
   
== Complete the On Boarding Process==
 
Other files may be copied under this directory if the project's configuration is more specific than the group configuration files, located under '''<HOME>/Lingoport_Data/L10nStreamlining/<group_name>/config''':
 
:Edit '''<code><HOME>/Lingoport_Data/L10nStreamlining/<group_name>/config/config_due_dates.xml</code>''': How the due dates are calculated.
 
:Edit '''<code><HOME>/Lingoport_Data/L10nStreamlining/<group_name>/config/config_l10n_vendor.properties</code>''': The way to send files to a vendor (Lingotek, WorldServer, or FTP configuration)
 
   
  +
Finally, update the '''<code><HOME>/Lingoport_Data/L10nStreamlining/config/config_email_global.xml</code>''' file and the following values.
 
  +
[[File:Build2.png|700px]]
<pre>
 
  +
<email-sender>sender@acme.com</email-sender>
 
  +
* You can have more than one file extension. For example, you may have .json file and .properties files in the same repository. '''For each file''' extension, fill in or modify the following fields:
<sender-password encrypted="false">agoodpassword</sender-password>
 
  +
** Extension: the resource file extension, for instance properties (without the '.'), json, resx, etc.
<!--This is the Dashboard URL used as the link destination for the email Log In button -->
 
  +
** Filename Pattern:
<dashboard-url>http://www.acme.com/dashboard</dashboard-url>
 
  +
*** The <code>*</code> (star) character represents the filename without the extension or the locales;
</pre>
 
  +
*** <code>l</code> represents the locale's language, for instance 'fr' for French
  +
*** <code>c</code> represents the locale's country, for instance 'FR' for France
  +
*** <code>v</code> represents the locale's variant. That's less frequent.
  +
** Use Pattern on Default Locale checkbox: Is the default locale named with its locale?
  +
*** Check this box if the default file is '''message_en_US.properties''' and the target locale will be '''messages_fr_FR.properties'''.
  +
*** Leave this box unchecked if the default locale file does not use the locale. If the default file is '''messages.properties''' and the target locale is to be '''messages_fr_FR.properties'''.
  +
** A few examples may help:
  +
*** <code>*_l_c_v</code>: If a target locale is '''fr_FR''' and '''messages_en.properties''' needs to be translated, the French file will be '''message_fr_FR.properties'''
  +
*** <code>*.l-c-v</code>: If a target locale is '''fr_FR''' and '''messages.resx''' needs to be translated, the French file will be '''messages.fr-FR.resx'''
  +
*** <code>*-l-c-v</code>: If a target locale is '''fr_FR''' and '''messages-en.json''' needs to be translated, the French file will be '''messages-fr-FR.json'''
  +
** File Location Pattern: Use this if the locales are in different directories. For example, the default locales could be in '''en/translation.json''' and the translated file in '''fr/translation.json'''. This pattern is the same as the Filename Pattern.
  +
** Use Location Pattern on Default Locale: Is the default locale in a folder with its locale?
  +
*** Check this box if the default location is '''locales_en/translation.json''' and the target locale is '''locales_fr/translation.json'''.
  +
*** Leave this box unchecked if the default location is '''locales/''' and the target locale is '''locales_fr/'''.
  +
  +
* The samples at the bottom of the Resource File Extensions box shows how the system is going to look for resource files. Verify that the resource files look like the example, otherwise errors could occur.
  +
* For the other entries, leave them defaulted unless told to change them.
  +
* If the resource files use the same names under different directories, for example, '''file1_en/translation.json''' and '''file2_en/translation.json''', make sure to check '''Send Unique Filenames''' under General Settings. Otherwise more than one translation kit will be sent out.
  +
  +
[[File:Build3.png|700px]]
  +
  +
* '''Directories''': Which directories and files to include and which ones to exclude as resource files.
  +
* '''Emails''': Who will be receiving emails for which notification
  +
  +
[[File:Build4.png|700px]]
  +
* The L10n Vendor can be configured in Jenkins, Manage Jenkins or in the project. Also a project can have a different value than other projects in the same group.
  +
  +
==== Post Build Actions ====
  +
* Nothing needs to be modified in this section
  +
  +
Select the '''Save''' button. '''Enable''' the project and '''Build Now'''. The Jenkins project should run and push the results to the Lingoport Dashboard.
   
 
For more information, see [[Project Configuration Files]]
 
For more information, see [[Project Configuration Files]]
  +
  +
= Verifying the Automation Job =
  +
  +
The Jenkins job should complete successfully and populate the Lingoport Dashboard.
  +
  +
To check if it completed successfully, select the project and look at the '''Build History'''
  +
  +
[[File:BuildHistory.jpg|500px]]
  +
  +
The indicator should look like a blue ball.
  +
  +
If it is a red ball, select it to go to '''Console Output''' and look for errors.
  +
  +
If it is a blue ball, select it and look for the output following "List of Target Files". It will look like the following:
  +
  +
resources/messages_de_DE.properties
  +
resources/messages_fr_FR.properties
  +
resources/messages_ja_JP.properties
  +
resources/messages_zh_CN.properties
  +
  +
Make sure the target files are correct. If not, you may want to change some of the on-boarding parameters, for instance the exclude directories if the list of target files includes test files or application configuration which should not be translated.
  +
  +
  +
  +
[[File:DashboardwFiles.png|700px]]
  +
  +
If the indicator is blue, then login to the Lingoport Dashboard and verify that the project is on the Dashboard. When you hover over the '''Next Send Translation Content''', a list of files to be translated will come up. Verify that they are the correct files to be translated.
  +
  +
If there are problems at this point, please contact '''support@lingoport.com''' for assistance.
  +
  +
== Set up the "send to translate" URL on the Dashboard ==
  +
Finally, if this is the first project on the Dashboard, the URL for Jenkins needs to be configured.
  +
# Select '''Administration''' on the top of the Dashboard.
  +
# Select '''Lingoport''' in the left side menu
  +
# Scroll down to '''Resource Manager''' and '''Jenkins URL'''
  +
# Enter the URL for Jenkins and '''Save'''.
  +
  +
[[File:JenkinsURL.png|600px]]
  +
  +
  +
Returning to the Resource Manager page, the '''Send to translate''' button should be populated in the '''Next Send Translation Content''' area. If there are "no files to prep and send for translation", the button will not appear. When this button is pressed, the Jenkins DashboardPrepKit job should be triggered and run. If that does not happen, contact ''support@lingoport.net''.
  +
  +
[[File:JenkinsURLSendToTranslate.png|600px]]
  +
  +
= Next Steps =
  +
  +
The following are optional helper jobs for LRM projects.
  +
  +
The Nightly job, checks the status of projects on a nightly basis and sends out notifications.
  +
  +
The PrepKit job will send the projects files to be translated. This can also be done from the Dashboard.
  +
  +
*[[On Boarding the Nightly Job]]
  +
*[[On Boarding the Prep Kit Job]]

Revision as of 15:02, 22 July 2020

Before On Boarding an LRM project

To on-board a project, a number of basic information is needed:

Code Repository Location

Users

  • The list of translation requesters. Those members to add to Jenkins so they have permission to trigger the Prep Kit job: username1@company.org, username2@companyname.com
  • The list of emails to be notified of the LRM actions: username1@company.org, username2@companyname.com, username3@companyname.com Note: All emails will also be used as Dashboard user identifiers.

Locales

  • The locale of the files created by developers and to be translated
  • The list of target translation locales
  • The pseudo-locale

Resource File types

  • Files types (for instance .properties, .resx, etc.). For more information on Resource file types and what is supported, please see Resource Files
  • Naming convention for the base file: for instance <filename>_en.properties or <filename>.properties
  • The directory structure for the resource files.
  • Translated file naming convention: for instance <filename>_fr_FR.properties
  • The files or directories which follow the above convention but should NOT be translated. For instance **/config*.properties

Localization Vendor

  • Who is the translation management system and how are the resource files going to be transferred to them?
  • Lingoport supports multiple vendors including Local vendors, FTP and SFTP, WorldServer, Lingotek, Systran and AWS Amazon Translate.

For more information about selecting a L10n Vendor for the project see Creating L10n Vendors

On Boarding the LRM Automation Job

Jenkins

Create the Jenkins job

To begin the on-boarding process, go to Jenkins, and select the 1-Automation Tab, and New Item

Automation tab new item.jpg

  • Make a copy of Lingoport.SampleLRM
  • Set the name of the new job to <groupname>.<projectname>
  • Make sure Add to current view is checked and select OK

Configure the Jenkins job

General

GeneralTab.png

  • General section: Select the GitHub project and fill in the Project url
  • Throttle Current Builds should be checked.
  • LRM Throttle should be checked.

Source Code Management

Source code management.jpg

Source Code Management section: Set up the Repository, Credentials, and Branch. For example if the Git repository is Acme/myproject and the code to work with is in the 'development' branch, then set the Repository URL to https://github.com/Acme/myproject.git. It is often easy to copy this from GitHub. For the branch, make sure to define it as */development. If the '*/' is left off, there will be problems.

Build Triggers

BuildTriggers.png

These are left unchecked by default, but the Build Periodically can be set to build on a repeating basis.

Build Environment

BuildEnvironment.png

Leave these unchecked.

Build

Build1.png

  • The Top Level Directory is defaulted to look at the whole repository. If the resource files are located under a resources directory, this can be changed to $WORKSPACE/resources .
  • Locales are defaulted to common values. Remove and add locales to get the appropriate translation languages. Pseudo locale defaults to 'eo'.
  • Copy Settings: Use this if you have a project already set up with a similar configuration. This can save configuration time. If there is no other project, just leave it unfilled.
  • Check the Send Unique File Names if you want the prep kit files to have unique file names. The default is not to send unique file names. If you have multiple file names that have the same name you may want to check the box. For more information see LRM Unique File Names.
  • Advanced: Detect Errors: It is unlikely you need to change the default settings.
    • Missed Translation Error: (Default: Unchecked) If you check this box, when a string is returned back from translation with the same text as the original one, an error will be raised. Be cautious with this check: Population for instance is the same word in English and in French.
    • Parameter Mismatch Error: (Default: Checked) Keep this check if you'd like to make sure that when a parameter(s) is (are) sent in a text, say "Hello {userName}", the same parameter(s) is (are) sent back untranslated. If the French translation is "Bonjour {nom d'utilisateur}", the 'userName' variable cannot be replaced at run time in the application.


Build2.png

  • You can have more than one file extension. For example, you may have .json file and .properties files in the same repository. For each file extension, fill in or modify the following fields:
    • Extension: the resource file extension, for instance properties (without the '.'), json, resx, etc.
    • Filename Pattern:
      • The * (star) character represents the filename without the extension or the locales;
      • l represents the locale's language, for instance 'fr' for French
      • c represents the locale's country, for instance 'FR' for France
      • v represents the locale's variant. That's less frequent.
    • Use Pattern on Default Locale checkbox: Is the default locale named with its locale?
      • Check this box if the default file is message_en_US.properties and the target locale will be messages_fr_FR.properties.
      • Leave this box unchecked if the default locale file does not use the locale. If the default file is messages.properties and the target locale is to be messages_fr_FR.properties.
    • A few examples may help:
      • *_l_c_v: If a target locale is fr_FR and messages_en.properties needs to be translated, the French file will be message_fr_FR.properties
      • *.l-c-v: If a target locale is fr_FR and messages.resx needs to be translated, the French file will be messages.fr-FR.resx
      • *-l-c-v: If a target locale is fr_FR and messages-en.json needs to be translated, the French file will be messages-fr-FR.json
    • File Location Pattern: Use this if the locales are in different directories. For example, the default locales could be in en/translation.json and the translated file in fr/translation.json. This pattern is the same as the Filename Pattern.
    • Use Location Pattern on Default Locale: Is the default locale in a folder with its locale?
      • Check this box if the default location is locales_en/translation.json and the target locale is locales_fr/translation.json.
      • Leave this box unchecked if the default location is locales/ and the target locale is locales_fr/.
  • The samples at the bottom of the Resource File Extensions box shows how the system is going to look for resource files. Verify that the resource files look like the example, otherwise errors could occur.
  • For the other entries, leave them defaulted unless told to change them.
  • If the resource files use the same names under different directories, for example, file1_en/translation.json and file2_en/translation.json, make sure to check Send Unique Filenames under General Settings. Otherwise more than one translation kit will be sent out.

Build3.png

  • Directories: Which directories and files to include and which ones to exclude as resource files.
  • Emails: Who will be receiving emails for which notification

Build4.png

  • The L10n Vendor can be configured in Jenkins, Manage Jenkins or in the project. Also a project can have a different value than other projects in the same group.

Post Build Actions

  • Nothing needs to be modified in this section

Select the Save button. Enable the project and Build Now. The Jenkins project should run and push the results to the Lingoport Dashboard.

For more information, see Project Configuration Files

Verifying the Automation Job

The Jenkins job should complete successfully and populate the Lingoport Dashboard.

To check if it completed successfully, select the project and look at the Build History

BuildHistory.jpg

The indicator should look like a blue ball.

If it is a red ball, select it to go to Console Output and look for errors.

If it is a blue ball, select it and look for the output following "List of Target Files". It will look like the following:

   resources/messages_de_DE.properties
   resources/messages_fr_FR.properties
   resources/messages_ja_JP.properties
   resources/messages_zh_CN.properties

Make sure the target files are correct. If not, you may want to change some of the on-boarding parameters, for instance the exclude directories if the list of target files includes test files or application configuration which should not be translated.


DashboardwFiles.png

If the indicator is blue, then login to the Lingoport Dashboard and verify that the project is on the Dashboard. When you hover over the Next Send Translation Content, a list of files to be translated will come up. Verify that they are the correct files to be translated.

If there are problems at this point, please contact support@lingoport.com for assistance.

Set up the "send to translate" URL on the Dashboard

Finally, if this is the first project on the Dashboard, the URL for Jenkins needs to be configured.

  1. Select Administration on the top of the Dashboard.
  2. Select Lingoport in the left side menu
  3. Scroll down to Resource Manager and Jenkins URL
  4. Enter the URL for Jenkins and Save.

JenkinsURL.png


Returning to the Resource Manager page, the Send to translate button should be populated in the Next Send Translation Content area. If there are "no files to prep and send for translation", the button will not appear. When this button is pressed, the Jenkins DashboardPrepKit job should be triggered and run. If that does not happen, contact support@lingoport.net.

JenkinsURLSendToTranslate.png

Next Steps

The following are optional helper jobs for LRM projects.

The Nightly job, checks the status of projects on a nightly basis and sends out notifications.

The PrepKit job will send the projects files to be translated. This can also be done from the Dashboard.