Difference between revisions of "Localyzer Projects"

From Lingoport Wiki
Jump to: navigation, search
(The Project Definition File)
(How can I tell if my Localyzer project is setup correctly?)
 
(34 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 
==Creating a project in Jenkins==
 
==Creating a project in Jenkins==
===New project===
 
LRM creates projects using a [[Terms_and_Definitions#projectdefn|Project Definition XML]] file that contains information about the resources and types for translation and then calling the [[LRM_Commands_Reference#Project_Commands| Create Project Command]].
 
   
  +
To create a new Localyzer project, follow [[On Boarding an LRM Project]] or [[On Boarding a project with both LRM and Globalyzer]] depending on the circumstances.
This is automated in Jenkins with the Lingoport plugin.
 
   
  +
== What is 'Missed Translation Error' and 'Parameter Mismatch Error'?==
[[File: JenkinsNewProject.jpg |600px]]
 
   
  +
When configuring an Localyzer project, there is an '''Advanced...''' button in the '''General Settings''' section. Clicking that will open a new section called '''Detect Errors'''.
To create a new project:
 
# Select '''New Item''' from the menu on the left
 
# In the dialog box, enter the name of the new project. The name must be of the form '''<group>.<project>''', where the group is the name associated with the LRM license. If you are unsure about the group name, a Jenkins job called '''<group>.LicenseExpiration''' should have been created at installation. Use that <group> name.
 
# In the ''Copy from'' box, enter '''Lingoport.SampleLRM''' or '''LingoportSampleLiteLRM'''.
 
   
  +
[[File:DetectErrors.jpg|700px]]
Note-a project can be created using '''either''' the project definition file directly on the command line or through the Jenkins LRM Plugin.
 
   
  +
This image shows the default values of '''Missed Translation Error''' and '''Parameter Mismatch Error'''.
===Configure the project===
 
'''Set the git directory'''
 
   
  +
'''Missed Translation Error''' is defaulted to be unset. This will trigger an error if the text in a translated file is the same as the text in the base file indicating that a translation may not have been done. Sometimes, however, a word or phrase is the same in more than one language.
[[File:GithubProject.jpg|600px]]
 
   
  +
'''Parameter Mismatch Error''' is defaulted to be set. Parameters can be defined in a values like:
Set the '''Project url''' to the URL for the GitHub repository with the code.
 
   
  +
<code>This is an {0} example of a string {1} with 2 parameters</code>
   
  +
If the translated string is returned without 2 parameters, then the error will be triggered.
Set the repository info and the branch name for the code.
 
   
  +
==How can I tell if my Localyzer project is setup correctly?==
'''Set up the Git Repository and the branch'''
 
   
  +
If the Localyzer Jenkins job fails when creating an Localyzer project, fix those failures before moving forward. If there are ERRORs in the Jenkins job output, but the job passes, debug those errors before moving forward.
[[File:JenkinsGitRepo.jpg |600px]]
 
 
Make sure that the '''Repository URL''' and the '''Branch Specifier''' are correct.
 
 
'''Set the General Settings and Locales'''
 
 
[[File:JenkinsLRMSettings.jpg |600px]]
 
 
The '''Top Level Directory''' is defaulted to $WORKSPACE. That means that LRM looks at the whole repository to find the resource files. This can be changed to look at only specific directories if you know where the resource files are kept.
 
 
The Send Unique Filenames checkbox is unchecked. Check this if the resource files are in the form:
 
 
folder1:
 
en_US.properties
 
folder2:
 
en_US.properties
 
 
There are two resource files called <code>en_US.properties</code>. These will be given unique filenames in the prep kit when sent to the translation system and then imported back to the correct location.
 
 
Any number of locales in various formats including LCID can be entered.
 
 
'''Set up the Resource File Extensions '''
 
 
[[File:JenkinsLRMResourceFileExtensions.jpg|600px]]
 
 
* In this example, the resource file type '''extension''' and '''parser type''' is 'properties' and 'rc'.
 
* We are using the default '''Parameter Regex Pattern'''.
 
* The '''Filename pattern''' is *_l_c_v which means that it expects filenames to be in the form 'something_xx_yy.properties'.
 
* Since '''Use Pattern on Default Locale''' is checked, LRM will look for the Filename pattern on the resource file for the default locale.
 
* '''File Location Pattern''' is empty, so all the resource files will be located in the same directory.
 
* '''Use Location Pattern on Default Locale''' is unchecked. It can't be checked if File Location Pattern is empty, but it tells what the directories will look like for the base and translated resources.
 
* Using these variables, the Jenkins job gives and example of the Base Resource File. It should be of the form <code>file1_en_us.properties</code> and the target resources will be of the form <code>file1_zh_hans_tw.properties</code>. If the resource files do not conform to these settings, errors and missed files might occur.
 
 
'''Set the directories and emails'''
 
 
[[File:JenkinsDirEmails.jpg|600px]]
 
 
* The '''Include Directory/File''' defaults to <code>**/**</code>. All the code in the repository is included. More directories or files can be included.
 
* The '''Exclude Directory Files''' have pre-filled entries for <code>**\*_eo.properties</code> (any pseudo-localized files), <code>**/LRMScans/**,**/GlobalyzerScans/**</code> and <code>**/sonar-project.properties</code> (all Lingoport files that don't have repos). More files can be excluded as needed.
 
* The email addresses need to be set up to indicate who should receive the various notifications. More than one address can be set for each notification.
 
 
'''Save''' the values and return to the Jenkins menu. '''Enable''' the project, if it is disabled and select '''Build Now''' from the menu to create the new project.
 
 
=== Check the Results ===
 
 
After the project builds, there is a red or blue indicator under the job's Build History. Select the indicator to see the Console Output for the newly created job.
 
 
[[File:BuildHistory.jpg|600px]]
 
 
The Console Output can be daunting, but there are a few things to verify.
 
 
[[File:FirstTimeBuild.jpg|600px]]
 
 
* Since this is the first time running, the '''Project Definition File''' and several other configuration files will be created.
 
* Verify that '''ERROR''' does not occur anywhere in the output.
 
* At the bottom of the the output, it should say '''ANALYSIS SUCCESSFUL''' and '''END UPDATE DASHBOARD'''.
 
 
At this point, the Lingoport Dashboard should have been updated with the project information.
 
 
If any of these criteria are not met, troubleshoot the project before moving forward.
 
 
=== The Project Definition File ===
 
LRM creates projects using a [[Terms_and_Definitions#projectdefn|Project Definition XML]] file that contains information about the resources and types for translation and then calling the [[LRM_Commands_Reference#Project_Commands| Create Project Command]]. Here is a typical .xml definition for projects with .properties files.
 
 
Note - a project should be created through the Jenkins LRM Plugin when making a Jenkins job.
 
 
<pre>
 
<lrmconf>
 
<model-version>5.1</model-version>
 
<project-name>NewLRMProject</project-name>
 
<project-desc/>
 
<group-name>Acme</group-name>
 
<top-level-dir>/var/lib/jenkins/jobs/Acme.NewLRMProject/workspace/</top-level-dir>
 
<send-unique-file-names>0</send-unique-file-names>
 
<detect-errors>
 
<missed-trans-error>0</missed-trans-error>
 
<parameter-mismatch-error>1</parameter-mismatch-error>
 
</detect-errors>
 
<default-locale>en_US</default-locale>
 
<pseudo-locale>eo</pseudo-locale>
 
<target-locales>
 
<locale>fr_FR</locale>
 
<locale>fr_CA</locale>
 
<locale>es_MX</locale>
 
</target-locales>
 
<resource-extensions>
 
<resource-extension>
 
<extension>properties</extension>
 
<parser-type>properties</parser-type>
 
<file-name-pattern>*_l_c_v</file-name-pattern>
 
<use-pattern-on-dflt-locale>1</use-pattern-on-dflt-locale>
 
<file-location-pattern/>
 
<use-location-pattern-on-dflt-locale>0</use-location-pattern-on-dflt-locale>
 
<base-file-encoding>UTF-8</base-file-encoding>
 
<localized-file-encoding>UTF-8</localized-file-encoding>
 
<parameter-regex-pattern><![CDATA[\{\w+\}|%[ds]]]></parameter-regex-pattern>
 
</resource-extension>
 
<resource-extension>
 
<extension>rc</extension>
 
<parser-type>rc</parser-type>
 
<file-name-pattern>*_l_c_v</file-name-pattern>
 
<use-pattern-on-dflt-locale>1</use-pattern-on-dflt-locale>
 
<file-location-pattern/>
 
<use-location-pattern-on-dflt-locale>0</use-location-pattern-on-dflt-locale>
 
<base-file-encoding>UTF-8</base-file-encoding>
 
<localized-file-encoding>UTF-8</localized-file-encoding>
 
<parameter-regex-pattern><![CDATA[%\d+]]></parameter-regex-pattern>
 
</resource-extension>
 
</resource-extensions>
 
<dirset>
 
<includes>
 
<include-dir-file>**/**</include-dir-file>
 
</includes>
 
<excludes>
 
<exclude-dir-file>**/bin/**</exclude-dir-file>
 
<exclude-dir-file>**/LRMScans/**</exclude-dir-file>
 
<exclude-dir-file>**/GlobalyzerScans/**</exclude-dir-file>
 
</excludes>
 
</dirset>
 
</lrmconf>
 
</pre>
 
 
==== Project Info ====
 
Project information includes the following:
 
* Project Name '''<code><project-name>NewLRMProject</project-name></code>''' - the repository name. This '''cannot be changed''' once the project is created.
 
* Project desc '''<code><project-desc/></code>'''- description, if any, for the project
 
* Group Name '''<code><group-name>Acme</group-name></code>''' - the owner of the repository. This '''cannot be changed''' once the project is created. In Jenkins, the job will be <group-name>.<project-name>
 
* Top Level Directory '''<code><top-level-dir>/var/lib/jenkins/jobs/Acme.NewLRMProject/workspace/</top-level-dir></code>''' - The top level directory is an absolute path. Usually it is something like '''<code>/var/lib/jenkins/jobs/Acme.NewLRMProject/workspace</code>'''. There are times where there is 1 repository for multiple projects. In this case the top level directory may be '''<code>/var/lib/jenkins/jobs/Acme.NewLRMProject/workspace/wileyProject1</code>'''. This value '''cannot be changed''' once the project is created.
 
 
==== Detect Errors Flags ====
 
These flags direct LRM on the types of errors to detect. The values are only shown in Jenkins in the Advanced section. In general, these are not modified from their default values.
 
* Missed Translations '''<code><missed-trans-error>0</missed-trans-error></code>''' - If set to ''1'' (true), then a ''Missed Translation'' error will occur if the base resource file text is the same as the translated file text. Usually this flag is set to '''0'''.
 
* Parameter Mismatch Error '''<code><parameter-mismatch-error>1</parameter-mismatch-error></code>'''- If set to ''1''(true) then a ''Parameter Mismatch'' error will occur if the parameters in the translated file do not match the base resource file parameters.
 
 
=== Locales ===
 
 
There are 4 types of java locales, that can be defined in a project:
 
* Track Back locale - The locale for the project's trackback files. See [[LRM_Commands_Reference#Create_Trackback_files| Trackback files ]]
 
* Pseudo Locale - The locale for the project's pseudo-locale files.
 
* Target locales - Must be exclusive of the project's base, trackback and pseudo-locales.
 
* Base Locale - The locale of the project's base resource file.
 
 
==== ProjectDefinition.xml example ====
 
The following defines ''br'' as the trackback locale, ''eo'' as the pseudo-locale, ''en'' as the base locale and ''fr_FR'', ''de_DE'' (case sensitive) as the target locales. The locales defined are in the format of a java locale (using underscores) and may not reflect the name of the resource files. The file name patterns are defined [[Supported_Resource_Bundles#Resource_Extensions|here]].
 
 
&lt;track-back-locale&gt;br&lt;/track-back-locale&gt;
 
&lt;pseudo-locale&gt;eo&lt;/pseudo-locale&gt;
 
&lt;target-locales&gt;
 
&lt;locale&gt;fr_FR&lt;/locale&gt;
 
&lt;locale&gt;de_DE&lt;/locale&gt;
 
&lt;/target-locales&gt;
 
&lt;default-locale&gt;en&lt;/default-locale&gt;
 
 
=== Resource Extensions ===
 
A LRM project can have multiple extensions defined. The information need per extension is:
 
* '''extension''' - this can be one of the [[Supported_Resource_Bundles#Standard_LRM_extensions|LRM standard extensions]] or a [[Supported_Resource_Bundles#Unique_Extensions|unique extension]].
 
* '''parser-type''' - A parser-type is required for unique extension.
 
* '''file name pattern''' - if applicable, this is the pattern for the name of project's resource files. For example, the translated file names ''file1_zh-Hans_HK.rc'', ''file1_fr.rc'' and ''file1_de-de.rc'' would have the pattern ''*_l-c_v''. Where ''l'' is the language, ''c'' is the country/script and ''v'' is the variant/region.
 
* '''use name pattern on default locale flag'''- If the base resource files use the file-name-pattern in their name then this flag is ''1'' otherwise ''0''. For example, the base resource file ''file1.rc'' does not have a pattern so the flag would be ''0''.
 
* '''file location pattern''' - if applicable, this is the pattern for the folders containing the resource files. These folder can be of 2 types:
 
** Language pattern - For example, ''*-l_c_v'' where the folder containing the English resource files could be ''values-en''.
 
** Microsoft LCID code pattern - For example, ''*_LCID'' where the folder containing the English(US) (en_us) resource files could be ''values_1033''.
 
:The location pattern may contain a wildcard for directory names. These are all valid location patterns:
 
:* ''*-l-c-v/*/*'' - directory depth is 3
 
:* ''*-l-c-v/some*folder'' - directory depth is 2
 
:* ''*_LCID/*/*/locales'' - directory depth is 4
 
:The location pattern '''can not''' contain wildcards for directory depth (**). These are '''invalid''' location patterns:
 
:* ''*-l-c-v/**'' (INVALID)
 
:* ''*-l-c-v/**/locales'' (INVALID)
 
* '''use location pattern on default locale flag''' - If the folder, containing the base resource files, use the pattern then this flag is ''1'' otherwise ''0''.
 
* '''encoding''' for the base and non-base resource files - Usually this will be UTF-8.
 
* '''parameter-regex-pattern''' - This defines the regex pattern for parameters such as formatting and messaging.
 
 
==== ProjectDefinition.xml example ====
 
The following will define the following directory structure.
 
 
src
 
: resources
 
:: ''file1_en-US.xml''
 
:: ''file1_fr-FR.xml''
 
:: ''file1_de-DE.xml''
 
: values
 
:: ''example.resx''
 
: values-1033
 
:: ''example.resx''
 
: values-1036
 
:: ''example.resx''
 
 
&lt;resource-extensions&gt;
 
&lt;resource-extension&gt;
 
&lt;extension&gt;resx&lt;/extension&gt;
 
&lt;file-name-pattern/&gt;
 
&lt;use-pattern-on-dflt-locale&gt;0&lt;/use-pattern-on-dflt-locale&gt;
 
&lt;file-location-pattern&gt;*-LCID&lt;/file-location-pattern&gt;
 
&lt;use-location-pattern-on-dflt-locale&gt;0&lt;/use-location-pattern-on-dflt-locale&gt;
 
&lt;base-file-encoding&gt;UTF-8&lt;/base-file-encoding&gt;
 
&lt;localized-file-encoding&gt;UTF-8&lt;/localized-file-encoding&gt;
 
&lt;parameter-regex-pattern&gt;&lt;![CDATA[%[ds]|%\d+\$[ds]|%\{\w+\}]]&gt;&lt;/parameter-regex-pattern&gt;
 
&lt;/resource-extension&gt;
 
&lt;resource-extension&gt;
 
&lt;extension&gt;xml&lt;/extension&gt;
 
&lt;file-name-pattern&gt;*_l-c-v&lt;/file-name-pattern&gt;
 
&lt;use-pattern-on-dflt-locale&gt;1&lt;/use-pattern-on-dflt-locale&gt;
 
&lt;file-location-pattern/&gt;
 
&lt;use-location-pattern-on-dflt-locale&gt;0&lt;/use-location-pattern-on-dflt-locale&gt;
 
&lt;base-file-encoding&gt;UTF-8&lt;/base-file-encoding&gt;
 
&lt;localized-file-encoding&gt;UTF-8&lt;/localized-file-encoding&gt;
 
&lt;parameter-regex-pattern&gt;&lt;![CDATA[%d|%s|%\d+\$s|%\d+\$d|%\{\w+\}]]&gt;&lt;/parameter-regex-pattern&gt;
 
&lt;/resource-extension&gt;
 
&lt;/resource-extensions&gt;
 
 
=== Include/Exclude Files/Directories ===
 
The include portion of the project definition can be used to include folder or files from the search for resource files. The default is all folders and files will be included.
 
 
The exclude portion of the project definition can be used to exclude folder or files from the search for resource files. The default is that no folders/files will be excluded.
 
 
==== ProjectDefinition.xml example ====
 
The following definition directs the search to only those resource files with ''src'' as a top-level directory. Any resource files under a ''test'' folder will be ignored as well as the ''config.properties'' file.
 
 
&lt;dirset&gt;
 
&lt;includes&gt;
 
&lt;include-dir-file&gt;**/src/**&lt;/include-dir-file&gt;
 
&lt;/includes&gt;
 
&lt;excludes&gt;
 
&lt;exclude-dir-file&gt;**/test/**&lt;/exclude-dir-file&gt;
 
&lt;exclude-dir-file&gt;**/config.properties&lt;/exclude-dir-file&gt;
 
&lt;/excludes&gt;
 
&lt;/dirset&gt;
 
 
---------------
 
<pre>
 
<lrmconf>
 
<model-version>5.1</model-version>
 
<project-name>NewLRMProject</project-name>
 
<project-desc/>
 
<group-name>Acme</group-name>
 
<top-level-dir>/var/lib/jenkins/jobs/Acme.NewLRMProject/workspace/</top-level-dir>
 
<send-unique-file-names>0</send-unique-file-names>
 
<detect-errors>
 
<missed-trans-error>0</missed-trans-error>
 
<parameter-mismatch-error>1</parameter-mismatch-error>
 
</detect-errors>
 
<default-locale>en_US</default-locale>
 
<pseudo-locale>eo</pseudo-locale>
 
<target-locales>
 
<locale>fr_FR</locale>
 
<locale>fr_CA</locale>
 
<locale>es_MX</locale>
 
</target-locales>
 
<resource-extensions>
 
<resource-extension>
 
<extension>properties</extension>
 
<parser-type>properties</parser-type>
 
<file-name-pattern>*_l_c_v</file-name-pattern>
 
<use-pattern-on-dflt-locale>1</use-pattern-on-dflt-locale>
 
<file-location-pattern/>
 
<use-location-pattern-on-dflt-locale>0</use-location-pattern-on-dflt-locale>
 
<base-file-encoding>UTF-8</base-file-encoding>
 
<localized-file-encoding>UTF-8</localized-file-encoding>
 
<parameter-regex-pattern><![CDATA[\{\w+\}|%[ds]]]></parameter-regex-pattern>
 
</resource-extension>
 
<resource-extension>
 
<extension>rc</extension>
 
<parser-type>rc</parser-type>
 
<file-name-pattern>*_l_c_v</file-name-pattern>
 
<use-pattern-on-dflt-locale>1</use-pattern-on-dflt-locale>
 
<file-location-pattern/>
 
<use-location-pattern-on-dflt-locale>0</use-location-pattern-on-dflt-locale>
 
<base-file-encoding>UTF-8</base-file-encoding>
 
<localized-file-encoding>UTF-8</localized-file-encoding>
 
<parameter-regex-pattern><![CDATA[%\d+]]></parameter-regex-pattern>
 
</resource-extension>
 
</resource-extensions>
 
<dirset>
 
<includes>
 
<include-dir-file>**/**</include-dir-file>
 
</includes>
 
<excludes>
 
<exclude-dir-file>**/bin/**</exclude-dir-file>
 
<exclude-dir-file>**/LRMScans/**</exclude-dir-file>
 
<exclude-dir-file>**/GlobalyzerScans/**</exclude-dir-file>
 
</excludes>
 
</dirset>
 
</lrmconf>
 
</pre>
 
 
*'''<code><project-name>NewLRMProject</project-name></code>''' - the name of the project
 
*'''<code><project-desc/></code>''' - the description of the project that was given
 
*'''<code><group-name>Acme</group-name></code>''' - the group for the project. In Jenkins, the job will be <group-name>.<project-name>
 
*'''<code><top-level-dir>/var/lib/jenkins/jobs/Acme.NewLRMProject/workspace/</top-level-dir></code>''' - The Jenkins jobs directory where the code repository will be cloned to for scanning.
 
*'''<code><send-unique-file-names>0</send-unique-file-names></code>''' -'0' indicates unchecked (false)
 
*'''<code><default-locale>en_US</default-locale></code>''' - the base resource (US English)
 
*'''<code><pseudo-locale>eo</pseudo-locale></code>''' - the locale to use for pseudo-localization.
 
*'''<code><target-locales></code>''' - the locales to translate to.
 
*'''<code><resource-extensions></code>''' - group of the resource extensions, like <code>.json, .properties, .xml</code>
 
*'''<code><resource-extension></code>''' - a stanza for one resource extension
 
**'''<code><extension>properties</extension></code>''' - the resource extension. LRM will look for files of the form '''<code><filename>.properties</code>'''
 
**'''<code><parser-type>properties</parser-type></code>''' - a file of the form '''<code><filename>.properties</code>''' will be parsed as a .properties file.
 
**'''<code><file-name-pattern>*_l_c_v</file-name-pattern></code>''' - the translated file will be of the <filename>_l_c_v.properties. For example '''<code><filename>_fr_FR.properties</code>''' or '''<code><filename>_zh_hans_tw.properties</code>''' or '''<code><filename>_no.properties</code>'''
 
**'''<code><use-pattern-on-dflt-locale>1</use-pattern-on-dflt-locale></code>''' -'1' (true) says that the default locale will have a locale pattern, like '''<code><filename>_en_US.properties</code>'''. If this were '0' (false), the base file would look like '''<code><filename>.properties</code>'''.
 
*'''<code><includes></code>''' - files to include to look for the resource files
 
*'''<code><excludes></code>''' - files that are not resource files.
 
 
==How can I tell if my LRM project is setup correctly?==
 
 
If the LRM Jenkins job fails when creating an LRM project, fix those failures before moving forward. If there are ERRORs in the Jenkins job output, but the job passes, debug those errors before moving forward.
 
 
 
Once an LRM project is set up, check that it looks correct on the Lingoport Dashboard. Select the project name from the main menu and then select the '''Dashboard''' pulldown and '''Resource Manager'''.
+
Once a Localyzer project is set up, check that it looks correct on the Lingoport Dashboard. Select the project name from the main menu. This takes you to the overview
   
[[File:DashboardResourceManager.jpg|800px]]
+
[[File:DashboardLiteLRM.png|700px]]
   
  +
Select the '''Lingoport''' pulldown and '''Localyzer''' or click on '''Localyzer Summary Report'''
On the Dashboard,
 
  +
on the Dashboard,
   
 
* There should not be any critical errors (they will be in red)
 
* There should not be any critical errors (they will be in red)
 
* The '''Default Locale''' should be set to the locale that was set up.
 
* The '''Default Locale''' should be set to the locale that was set up.
 
* The '''Locale''' list should be the same set up in Jenkins.
 
* The '''Locale''' list should be the same set up in Jenkins.
* The numbre of '''Base Resource Files''' should be resonable.
+
* The number of '''Base Resource Files''' should be resonable.
* If you hover the mouse over '''Next Prep Kit Content''', a list of resource files will come up (as long as there are resource files to prep).
+
* If you hover the mouse over '''Next Send Translation Content''', a list of resource files will come up (as long as there are resource files to prep).
   
The '''project inspect''' reports are created upon creation of [[Terms_and_Definitions#LRMProject|LRM project]]. Otherwise, they can be run again from a command line:
+
The '''project inspect''' reports are created upon creation of [[Terms_and_Definitions#LRMProject|Localyzer project]]. Otherwise, they can be run again from a command line:
 
 
 
<pre>
 
<pre>
Line 358: Line 49:
 
</pre>
 
</pre>
   
This command generates reports in your reports folder for ''each locale'' specified in the [[Terms_and_Definitions#projectdefn|project definition file]]. Each report shows the location of the [[Terms_and_Definitions#resourcefile|resource files]], the number of files to be translated, and the file names.
+
This command generates reports in your reports folder for ''each locale'' specified in the project. Each report shows the location of the [[Terms_and_Definitions#resourcefile|resource files]], the number of files to be translated, and the file names.
   
 
<pre>
 
<pre>
Line 392: Line 83:
 
</pre>
 
</pre>
   
You may need to edit your project definition. See [[Supported_Resource_Bundles#Lingoport_Resource_Manager_Configuration|Project Definition]] for information about configuring a LRM project.
+
You may need to edit your project definition. See [[Supported_Resource_Bundles#Lingoport_Resource_Manager_Configuration|Project Definition]] for information about configuring a Localyzer project.
   
 
==Why, when I create a project, do I get an error stating that no resource files were found (Error 500)?==
 
==Why, when I create a project, do I get an error stating that no resource files were found (Error 500)?==
   
 
This is a pretty common error and generally easy to fix.
 
This is a pretty common error and generally easy to fix.
It sounds as if your project is not setup correctly. Check the Project Definition File used to create the project or open the Jenkins job Configuration.
+
Your project may not setup correctly. Check the Project Definition File used to create the project or open the Jenkins job Configuration. This error may also occur if you are using transforms. If that is the case, this may not be a problem and check if transforms ran successfully and the Dashboard looks correct.
  +
  +
In Jenkins, open the job configuration and scroll down to '''Resource File Extensions'''.
  +
  +
[[File:Build2.png|700px]]
  +
  +
The example in the extension shows what format of file that Localyzer is looking for. If the resource files do no look like this, then modify this section and see if the error goes away.
   
 
On the command line to check the Project Definition File:
 
On the command line to check the Project Definition File:
Line 403: Line 100:
   
 
The items to check (with the project definition file xml and the Jenkins configuration name) are:
 
The items to check (with the project definition file xml and the Jenkins configuration name) are:
* '''<top-level-dir>''' or '''Top Level Directory'''- this is the path to the source code (resource files) for the LRM Project. For example if you have <code>$WORKSPACE/resources</code> or <code><top-level-dir>/var/lib/jenkins/jobs/CET.Smoke/workspace/resources</top-level-dir></code> but the resource files are in <code>/var/lib/jenkins/jobs/CET.Smoke/workspace'''/locales'''</code>, then LRM will not find the resource files.
+
* '''<top-level-dir>''' or '''Top Level Directory'''- this is the path to the source code (resource files) for the Localyzer Project. For example if you have <code>$WORKSPACE/resources</code> or <code><top-level-dir>/var/lib/jenkins/jobs/CET.Smoke/workspace/resources</top-level-dir></code> but the resource files are in <code>/var/lib/jenkins/jobs/CET.Smoke/workspace'''/locales'''</code>, then Localyzer will not find the resource files.
 
<pre>
 
<pre>
 
<top-level-dir>/var/lib/jenkins/path/to/my/resource_files</top-level-dir>
 
<top-level-dir>/var/lib/jenkins/path/to/my/resource_files</top-level-dir>
Line 409: Line 106:
 
* Patterns for your resource file names:
 
* Patterns for your resource file names:
 
** '''<file-name-pattern>...</file-name-pattern>''' or '''Filename Pattern''' - verify that the separator characters are correct for your localized resource file names. For example, if your French Canadian localized resource file names are of the form <code>myResources_fr_CA.properties</code> then set the following: <pre><file-name-pattern>*_l_c_v</file-name-pattern></pre>
 
** '''<file-name-pattern>...</file-name-pattern>''' or '''Filename Pattern''' - verify that the separator characters are correct for your localized resource file names. For example, if your French Canadian localized resource file names are of the form <code>myResources_fr_CA.properties</code> then set the following: <pre><file-name-pattern>*_l_c_v</file-name-pattern></pre>
** '''<use-pattern-on-dflt-locale>...</use-pattern-on-dflt-locale>''' or '''Use Pattern on Default Locale''' - Set to "1" (or checked) (true), it indicates you will use base resource files that include the default locale in their filenames. For example, if the default locale is set to "en_US" and the file-name-pattern for "properties" resource files is set to "*_l_c_v", then LRM will look for '''<code>myResources_en_US.properties</code>''' when creating a kit of files to be sent out for translation. If set to "0" (or unchecked) (false), then LRM will look for base resource files that do not have the default locale as part of their filenames. Using our "properties" file example, LRM will look for '''<code>myResources.properties</code>''' when creating files for translation.<pre><use-location-pattern-on-dflt-locale>0</use-location-pattern-on-dflt-locale></pre>
+
** '''<use-pattern-on-dflt-locale>...</use-pattern-on-dflt-locale>''' or '''Use Pattern on Default Locale''' - Set to "1" (or checked) (true), it indicates you will use base resource files that include the default locale in their filenames. For example, if the default locale is set to "en_US" and the file-name-pattern for "properties" resource files is set to "*_l_c_v", then Localyzer will look for '''<code>myResources_en_US.properties</code>''' when creating a kit of files to be sent out for translation. If set to "0" (or unchecked) (false), then Localyzer will look for base resource files that do not have the default locale as part of their filenames. Using our "properties" file example, Localyzer will look for '''<code>myResources.properties</code>''' when creating files for translation.<pre><use-location-pattern-on-dflt-locale>0</use-location-pattern-on-dflt-locale></pre>
 
** '''<file-location-pattern>...</file-location-pattern>''' or '''File Location Pattern''' - verify that the separator characters are correct. For example, if it is set to *-l-c-v and the base resource files are in a '''<code>files</code>''' directory then the translated files will go into '''<code>files-fr-FR</code>''' for a french locale. If <code><use-location-on-dflt-locale>1</use-location-on-dflt-locale></code> or Use Location Pattern on Default Locale is checked (true), then the base resource files need to be in a folder that looks like '''<code>files-en-US</code>'''.
 
** '''<file-location-pattern>...</file-location-pattern>''' or '''File Location Pattern''' - verify that the separator characters are correct. For example, if it is set to *-l-c-v and the base resource files are in a '''<code>files</code>''' directory then the translated files will go into '''<code>files-fr-FR</code>''' for a french locale. If <code><use-location-on-dflt-locale>1</use-location-on-dflt-locale></code> or Use Location Pattern on Default Locale is checked (true), then the base resource files need to be in a folder that looks like '''<code>files-en-US</code>'''.
 
* '''<include-dir>...</include-dir>''' or '''Include Directories''' - verify that the value is a valid Ant formatting directive. If there is no value then only those files directly under the ''base-src-dir'' + ''top-level-dir'' will be searched; no recursive search will occur. To include all folders under the full path, enter a value of '''**/**'''. You can then exclude other folders by adding additional exclude Ant directives. Format for ''<include-dir>'' is: <pre><include-dir>**/**</include-dir></pre>
 
* '''<include-dir>...</include-dir>''' or '''Include Directories''' - verify that the value is a valid Ant formatting directive. If there is no value then only those files directly under the ''base-src-dir'' + ''top-level-dir'' will be searched; no recursive search will occur. To include all folders under the full path, enter a value of '''**/**'''. You can then exclude other folders by adding additional exclude Ant directives. Format for ''<include-dir>'' is: <pre><include-dir>**/**</include-dir></pre>
Line 416: Line 113:
 
== Why do I get an error "All active prep kits must be completed before changes can be made" when updating the Jenkins job?==
 
== Why do I get an error "All active prep kits must be completed before changes can be made" when updating the Jenkins job?==
   
LRM will error if Prep Kit jobs have been sent, but not returned and the Jenkins job is changed. For example, if the target locales are modified before all prep kits are returned, the Jenkins job will fail with the error:
+
Localyzer will error if Prep Kit jobs have been sent, but not returned and the Jenkins job is changed. For example, if the target locales are modified before all prep kits are returned, the Jenkins job will fail with the error:
   
 
<pre>
 
<pre>
Line 424: Line 121:
   
 
Once all outstanding prep kits have returned, then things can be modified in the job.
 
Once all outstanding prep kits have returned, then things can be modified in the job.
  +
  +
== How to on board an Android project with strings.xml file==
  +
  +
Android resource files are often called 'strings.xml' and exist in many 'values' directories. On boarding this type of project can be slightly different than other resource file types. Your resource files may look like:
  +
  +
.../memories/src/main/res/values/strings.xml
  +
.../shared/src/main/res/values/strings.xml
  +
.../family/src/main/res/values/strings.xml
  +
  +
Since we have the same filename in several directories, '''Send Unique Filenames''' should be selected so that all the resource files are sent in one kit and not several of them.
  +
  +
[[File:ResourceFileExtensionXml.png|700px]]
  +
  +
Select the xml '''Resource Extension Configuration''', removing the other extensions. Configure as shown in the picture. Files will be returned like: <code>.../values_de_DE/strings.xml</code>.
  +
  +
[[File:IncludeDirXml.png|700px]]
  +
  +
In the '''Directories''' section, '''Include Directory/File''', enter '''**/strings.xml'''. This way, only 'strings.xml' files will be analyzed and not other xml-type files which may not be resource files.
  +
  +
'''Save''' and '''Build Now''' the new project.
  +
  +
[[File:AndroidStringsDashboard.png|700px]]
  +
  +
The Dashboard will be created for the new project and the Resource Manager page should reflect the project and show the files that will be sent for translation. If the Dashboard does not look correct or shows the wrong files, return to the Jenkins job and check the configuration.
  +
  +
== How to on board an IOS project with .strings resource files ==
  +
  +
Resource files for IOS projects often look like <code>.../en.lproj/<filename>.strings</code> or <code>.../en_US.lproj/<filename>.strings</code>.
  +
  +
To on board a .strings project, create a new project. Set up the repository, branch, and the locales to be translated.
  +
  +
[[File:ResourceFileExtensionStrings.png|700px]]
  +
  +
Select the strings '''Resource Extension Configuration''', removing the other extensions. Configure as shown in the picture. Note the example shown, it should match the form of the translated files.
  +
  +
'''Save''' and '''Build Now''' the new project.
  +
  +
The Dashboard will be created for the new project and the Resource Manager page should reflect the project and show the files that will be sent for translation. If the Dashboard does not look correct or shows the wrong files, return to the Jenkins job and check the console output and the configuration.
  +
  +
== How to add or update an Localyzer license from the command line ==
  +
'''Using the command line is not recommended. Work with Jenkins and its jobs, whenever possible.''' The '''ManageLicenses''' job will add or update a license.
  +
  +
If you'd rather use the command line instead of the Jenkins ManageLicenses job, for instance in a script, use the <code>--create-group</code> command on lrm-cli.jar command line. Using the command line is not recommende. The ManageLicenses Jenkins job is more reliable. You need to have requested a license key for the new group from Lingoport before creating the group. For instance:
  +
  +
> java -jar $LRM_HOME/lrm-cli.jar --create-group --group-name group1 --date "2016-06-01" --number-of-licenses 10 --license-key <license-key>
  +
  +
To update the license of an existing group, used the <code>--update-license</code> command on lrm-cli.jar command line. You need to have requested a license key for the new group from Lingoport before updating the group.For instance:
  +
  +
> java -jar $LRM_HOME/lrm-cli.jar --update-license --group-name group1 --date "2016-06-01" -number-of-licenses 10 --license-key <license-key>
  +
  +
Once a group has been created, the database and the file structure are updated. The Jenkins LicenseExpiration job should be created and it should show the change to the license.
  +
  +
== How to change a project that has prep kits assigned ==
  +
  +
Sometimes, changes and updates need to be made to projects and when changed, there is a error about "Prep kits have been assigned" and the change cannot be made. It can be done, but it takes a few steps.
  +
  +
1) Note the last prep kit number sent so you can tell the projects to make sure they delete those. Otherwise, there may be a collision in the naming of the projects in the translation management system.
  +
  +
2) Go to the TTY console as user 'jenkins' and delete the Localyzer project, something like:
  +
  +
lrm --deleteProject -gn <group name> -pn <project name>
  +
  +
and type 'yes' when prompted. For a Jenkins project called Acme.myproject, this would look like:
  +
  +
lrm --deleteProject -gn Acme -pn myproject
  +
  +
3) Return to Jenkins and make the changes to the project and 'Build' it.
  +
  +
4) From the TTY console, check which files are being handled with:
  +
  +
lrm -gn <group name> -pn <project name> -r -pi
  +
for example:
  +
lrm -gn Acme -pn myproject -r -pi
  +
  +
and look at the report files that are generated.
  +
  +
5) Adjust your Jenkins on-boarding accordingly and redo 4 & 5 until satisfied :)

Latest revision as of 19:23, 26 August 2022

Creating a project in Jenkins

To create a new Localyzer project, follow On Boarding an LRM Project or On Boarding a project with both LRM and Globalyzer depending on the circumstances.

What is 'Missed Translation Error' and 'Parameter Mismatch Error'?

When configuring an Localyzer project, there is an Advanced... button in the General Settings section. Clicking that will open a new section called Detect Errors.

DetectErrors.jpg

This image shows the default values of Missed Translation Error and Parameter Mismatch Error.

Missed Translation Error is defaulted to be unset. This will trigger an error if the text in a translated file is the same as the text in the base file indicating that a translation may not have been done. Sometimes, however, a word or phrase is the same in more than one language.

Parameter Mismatch Error is defaulted to be set. Parameters can be defined in a values like:

This is an {0} example of a string {1} with 2 parameters

If the translated string is returned without 2 parameters, then the error will be triggered.

How can I tell if my Localyzer project is setup correctly?

If the Localyzer Jenkins job fails when creating an Localyzer project, fix those failures before moving forward. If there are ERRORs in the Jenkins job output, but the job passes, debug those errors before moving forward.

Once a Localyzer project is set up, check that it looks correct on the Lingoport Dashboard. Select the project name from the main menu. This takes you to the overview

DashboardLiteLRM.png

Select the Lingoport pulldown and Localyzer or click on Localyzer Summary Report on the Dashboard,

  • There should not be any critical errors (they will be in red)
  • The Default Locale should be set to the locale that was set up.
  • The Locale list should be the same set up in Jenkins.
  • The number of Base Resource Files should be resonable.
  • If you hover the mouse over Next Send Translation Content, a list of resource files will come up (as long as there are resource files to prep).

The project inspect reports are created upon creation of Localyzer project. Otherwise, they can be run again from a command line:

> java -jar lrm-cli.jar -r -pi -pn <ProjectName> -gn <GroupName>
> java -jar lrm-cli.jar --report --project-inspect --project-name <ProjectName> --group-name <GroupName>
The following reports were generated:
/var/lib/jenkins/Lingoport_Data/LRM/<GroupName>/reports/<ProjectName>/ProjectInspect_en_us.xml
/var/lib/jenkins/Lingoport_Data/LRM/<GroupName>/reports/<ProjectName>/ProjectInspect_de_de.xml
/var/lib/jenkins/Lingoport_Data/LRM/<GroupName>/reports/<ProjectName>e/ProjectInspect_fr_fr.xml
/var/lib/jenkins/Lingoport_Data/L10nStreamlining/<GroupName>/projects/<ProjectName>/reports/project_inspect_files.txt
Process completed successfully.

This command generates reports in your reports folder for each locale specified in the project. Each report shows the location of the resource files, the number of files to be translated, and the file names.

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<LRMProjectInspectResults projectID="217" projectName="YourProjectName" topLevelDirector
y="/location/of/files/resources">
  <locale>en_US</locale>
  <isBaseLocale>true</isBaseLocale>
  <nbFiles>3</nbFiles>
  <results>
    <result file="/file1.json"/>
    <result file="/file2.properties"/>
    <result file="/file3.rjs"/>
  </results>
</LRMProjectInspectResults>

In addition, file project_inspect_files.txt is created listing all of the absolute paths of the corresponding target files for each base resource file. The following is an example where:

  • The group name is GroupName
  • The project name is YourProjectName
  • The top level source directory defined in the project definition is location/of/files/resources
  • In the project definition, extension json has a name of pattern of *.l_c_v. The default locale files for this extension does not use the name pattern. There is no location pattern.
  • In the project definition, extension properties has a name of pattern of *_l_c_v. The default locale files for this extension does not use the name pattern. There is no location pattern.
  • In the project definition, extension rjs has a name of pattern of *-l-c-v. The default locale files for this extension does not use the name pattern. There is no location pattern.
/var/lib/jenkins/jobs/GroupName.YourProjectName/workspace/location/of/files/resources/file1.fr_fr.json
/var/lib/jenkins/jobs/GroupName.YourProjectName/workspace/location/of/files/resources/file1.es_mx.json
/var/lib/jenkins/jobs/GroupName.YourProjectName/workspace/location/of/files/resources/file2_fr_fr.properties
/var/lib/jenkins/jobs/GroupName.YourProjectName/workspace/location/of/files/resources/file2_es_mx.properties
/var/lib/jenkins/jobs/GroupName.YourProjectName/workspace/location/of/files/resources/file3-fr-fr.rjs
/var/lib/jenkins/jobs/GroupName.YourProjectName/workspace/location/of/files/resources/file3-es-mx.rjs

You may need to edit your project definition. See Project Definition for information about configuring a Localyzer project.

Why, when I create a project, do I get an error stating that no resource files were found (Error 500)?

This is a pretty common error and generally easy to fix. Your project may not setup correctly. Check the Project Definition File used to create the project or open the Jenkins job Configuration. This error may also occur if you are using transforms. If that is the case, this may not be a problem and check if transforms ran successfully and the Dashboard looks correct.

In Jenkins, open the job configuration and scroll down to Resource File Extensions.

Build2.png

The example in the extension shows what format of file that Localyzer is looking for. If the resource files do no look like this, then modify this section and see if the error goes away.

On the command line to check the Project Definition File:

> vi /var/lib/jenkins/Lingoport_Data/LRM/<group-name>/reports/<project-name>/ProjectDefinition.xml

The items to check (with the project definition file xml and the Jenkins configuration name) are:

  • <top-level-dir> or Top Level Directory- this is the path to the source code (resource files) for the Localyzer Project. For example if you have $WORKSPACE/resources or <top-level-dir>/var/lib/jenkins/jobs/CET.Smoke/workspace/resources</top-level-dir> but the resource files are in /var/lib/jenkins/jobs/CET.Smoke/workspace/locales, then Localyzer will not find the resource files.
<top-level-dir>/var/lib/jenkins/path/to/my/resource_files</top-level-dir>
  • Patterns for your resource file names:
    • <file-name-pattern>...</file-name-pattern> or Filename Pattern - verify that the separator characters are correct for your localized resource file names. For example, if your French Canadian localized resource file names are of the form myResources_fr_CA.properties then set the following:
      <file-name-pattern>*_l_c_v</file-name-pattern>
    • <use-pattern-on-dflt-locale>...</use-pattern-on-dflt-locale> or Use Pattern on Default Locale - Set to "1" (or checked) (true), it indicates you will use base resource files that include the default locale in their filenames. For example, if the default locale is set to "en_US" and the file-name-pattern for "properties" resource files is set to "*_l_c_v", then Localyzer will look for myResources_en_US.properties when creating a kit of files to be sent out for translation. If set to "0" (or unchecked) (false), then Localyzer will look for base resource files that do not have the default locale as part of their filenames. Using our "properties" file example, Localyzer will look for myResources.properties when creating files for translation.
      <use-location-pattern-on-dflt-locale>0</use-location-pattern-on-dflt-locale>
    • <file-location-pattern>...</file-location-pattern> or File Location Pattern - verify that the separator characters are correct. For example, if it is set to *-l-c-v and the base resource files are in a files directory then the translated files will go into files-fr-FR for a french locale. If <use-location-on-dflt-locale>1</use-location-on-dflt-locale> or Use Location Pattern on Default Locale is checked (true), then the base resource files need to be in a folder that looks like files-en-US.
  • <include-dir>...</include-dir> or Include Directories - verify that the value is a valid Ant formatting directive. If there is no value then only those files directly under the base-src-dir + top-level-dir will be searched; no recursive search will occur. To include all folders under the full path, enter a value of **/**. You can then exclude other folders by adding additional exclude Ant directives. Format for <include-dir> is:
    <include-dir>**/**</include-dir>
  • <exclude-dir-file>...</exclude-dir-file> or Exclude Directory Files - verify that the value is a valid Ant formatting directive and that you're not excluding valid files/folders. Format for <exclude-dir> is:
    <exclude-dir-file>...</exclude-dir-file>

Why do I get an error "All active prep kits must be completed before changes can be made" when updating the Jenkins job?

Localyzer will error if Prep Kit jobs have been sent, but not returned and the Jenkins job is changed. For example, if the target locales are modified before all prep kits are returned, the Jenkins job will fail with the error:

ERROR 2019-06-20 20:55:07,128 com.lingoport.lrm.controller.kitrootconfig.pipeline.stage.CompareExistingLocalesWithXMLStage.addError: Error 230: Cannot delete locales for project `NewLRMProject`. All active prep kits must be completed before changes can be made.
Process failed. See error log in `/var/lib/jenkins/Lingoport_Data/LRM/<group-name>/logs/lrmerror.log` for details.

Once all outstanding prep kits have returned, then things can be modified in the job.

How to on board an Android project with strings.xml file

Android resource files are often called 'strings.xml' and exist in many 'values' directories. On boarding this type of project can be slightly different than other resource file types. Your resource files may look like:

.../memories/src/main/res/values/strings.xml
.../shared/src/main/res/values/strings.xml 
.../family/src/main/res/values/strings.xml

Since we have the same filename in several directories, Send Unique Filenames should be selected so that all the resource files are sent in one kit and not several of them.

ResourceFileExtensionXml.png

Select the xml Resource Extension Configuration, removing the other extensions. Configure as shown in the picture. Files will be returned like: .../values_de_DE/strings.xml.

IncludeDirXml.png

In the Directories section, Include Directory/File, enter **/strings.xml. This way, only 'strings.xml' files will be analyzed and not other xml-type files which may not be resource files.

Save and Build Now the new project.

AndroidStringsDashboard.png

The Dashboard will be created for the new project and the Resource Manager page should reflect the project and show the files that will be sent for translation. If the Dashboard does not look correct or shows the wrong files, return to the Jenkins job and check the configuration.

How to on board an IOS project with .strings resource files

Resource files for IOS projects often look like .../en.lproj/<filename>.strings or .../en_US.lproj/<filename>.strings.

To on board a .strings project, create a new project. Set up the repository, branch, and the locales to be translated.

ResourceFileExtensionStrings.png

Select the strings Resource Extension Configuration, removing the other extensions. Configure as shown in the picture. Note the example shown, it should match the form of the translated files.

Save and Build Now the new project.

The Dashboard will be created for the new project and the Resource Manager page should reflect the project and show the files that will be sent for translation. If the Dashboard does not look correct or shows the wrong files, return to the Jenkins job and check the console output and the configuration.

How to add or update an Localyzer license from the command line

Using the command line is not recommended. Work with Jenkins and its jobs, whenever possible. The ManageLicenses job will add or update a license.

If you'd rather use the command line instead of the Jenkins ManageLicenses job, for instance in a script, use the --create-group command on lrm-cli.jar command line. Using the command line is not recommende. The ManageLicenses Jenkins job is more reliable. You need to have requested a license key for the new group from Lingoport before creating the group. For instance:

 > java -jar $LRM_HOME/lrm-cli.jar --create-group --group-name group1 --date "2016-06-01" --number-of-licenses 10 --license-key <license-key>

To update the license of an existing group, used the --update-license command on lrm-cli.jar command line. You need to have requested a license key for the new group from Lingoport before updating the group.For instance:

 > java -jar $LRM_HOME/lrm-cli.jar --update-license --group-name group1 --date "2016-06-01" -number-of-licenses 10 --license-key <license-key>

Once a group has been created, the database and the file structure are updated. The Jenkins LicenseExpiration job should be created and it should show the change to the license.

How to change a project that has prep kits assigned

Sometimes, changes and updates need to be made to projects and when changed, there is a error about "Prep kits have been assigned" and the change cannot be made. It can be done, but it takes a few steps.

1) Note the last prep kit number sent so you can tell the projects to make sure they delete those. Otherwise, there may be a collision in the naming of the projects in the translation management system.

2) Go to the TTY console as user 'jenkins' and delete the Localyzer project, something like:

   lrm --deleteProject -gn <group name> -pn <project name>

and type 'yes' when prompted. For a Jenkins project called Acme.myproject, this would look like:

   lrm --deleteProject -gn Acme -pn myproject

3) Return to Jenkins and make the changes to the project and 'Build' it.

4) From the TTY console, check which files are being handled with:

   lrm -gn <group name> -pn  <project name> -r -pi

for example:

   lrm -gn Acme -pn  myproject -r -pi

and look at the report files that are generated.

5) Adjust your Jenkins on-boarding accordingly and redo 4 & 5 until satisfied :)