Prep Kits

From Lingoport Wiki
Revision as of 17:06, 27 June 2019 by Rdaly (talk | contribs) (When I prep a kit, what files are created?)
Jump to: navigation, search

What is a Prep Kit?

A Prep Kit is what we call the set of resource files that are gathered by LRM, sent out for translation, and then integrated back into the source code tree once translated. LRM sets and tracks versions of a Kit so that, as your developers add strings to the base resource files, you can prepare and send out the Kit multiple times. There are a variety of checks that LRM makes on the base resource files to ensure that they are correct, as well as tracking the progress and correctness of the translated resource files.

Note: If your project contains translated files, then when prepping your first kit, LRM will not be able to detect if a text change has occurred within a resource string. It will, however, be able to detect added keys or missed translations.

How do I prep a kit and receive it back from translation?

There is a Jenkins job on-boarded. The Lingoport Dashboard is populated and the L10n Vendor has been configured. A Prep Kit can be sent. At this point, the Lingoport Dashboard for the project looks something like:

BeforePrepKit.jpg


  • Last Sent Kit is 0 since no prep kits have been sent yet.
  • All the Locales are at 0 Percent Complete since we only have a base resource file.
  • Under Next Prep Kit Content, it shows that we are sending 1 file that has 62 keys and 82 words.
  • Since we have not sent anything yet, there is nothing in Prep Kit Due Dates

To send a prep kit, either select the Prep Kit button on the Dashboard (this runs the DashboardPrepKit job in Jenkins) or run the PrepKit job in Jenkins called <group>.<project>-PrepKit.

When everything is successful, the Dashboard should look like:

AfterPrepKit.jpg

  • Last Sent Kit is 1 since we sent Prep Kit #1 to be translated.
  • All the Locales are at still at 0 Percent Complete since we have not gotten the translations back, but there is a note: There are outstanding prep kits for this locale
  • Under Next Prep Kit Content, there is nothing because everything at this point has been sent.
  • Prep Kit Due Dates shows 1 file for each locale was sent and when it is due back.

For the persons that were designated in the Jenkins job to get notifications of prep kits, an email will be sent. When the prep kit is successfully sent out, the email will be an Upload Success. There will be one email for each locale.

UploadSuccess.jpg

This shows just the German (de_DE) locale.

  • Note that it shows CET.NOAA.1.de_DE that is <group-name>.<project-name>.<prep kit number>.<locale>
  • It shows where to find the files on the local machine that were sent to translation.
  • It lists the files sent to be translated to German. In this example, there is only 1 file.


When the files have been translated, they are put in a special directory and Jenkins routinely checks for files to be downloaded. The files are downloaded and and then the translated files are pushed to the repository. To do this manually, invoke the FTPDownload Jenkins job if using FTP. Once the files have been downloaded, the Dashboard will update to reflect the changes.

AfterImport.jpg

  • Next Prep Kit Content and Prep Kit Due Dates are empty. All files have been translated.
  • Each Locale shows 100% Complete with 0 Files, Keys or Words Remaining.

The code repository will have the translated files.

What is done when a prep kit is generated?

After a LRM project is created and correctly identifies the base and target resource files, the project needs to be baselined. When the project is baselined, the base resource files are in the system. If any changes happen to them afterward, this will be picked up in the prep kit. This is done in the Jenkins Nightly job or it can be done on the command line.

 > java -jar /var/lib/jenkins/lingoport/lrm-server-<version>/lrm-cli.jar --create-baseline --project-name <project-name> -- group-name <group-name>  

LRM is able to detect modified text as well as missing keys and missing files.

For example let's say you have a project with:

  • 3 locales: en_US (base locale), fr_FR and de_DE.
  • 2 base files
    • file1_en_US.properties (100 keys)
    • file2_en_US.properties
  • 1 fr_FR file with a missing key
    • file1_fr_FR.properties (99 keys)
  • 1 de_DE with 2 missing keys
    • file1_de_DE.properties (98 keys)


Create a prep kit for the new LRM project by selecting the Prep Kit button on the Lingoport Dashboard or by running the Jenkins <project-name>-PrepKit job.

> java -jar lrm-cli.jar -pk -pn ProjectName -gn GroupName
The following reports were generated:
/var/lib/jenkins/Lingoport_Data/LRM/GroupName/prep_kit/ProjectName/PREP_KIT_1/BaseFileSearch_de_de.xml
/var/lib/jenkins/Lingoport_Data/LRM/GroupName/prep_kit/ProjectName/PREP_KIT_1/BaseFileSearch_fr_fr.xml
/var/lib/jenkins/Lingoport_Data/LRM/GroupName/prep_kit/ProjectName/PREP_KIT_1/FilesToPrep.xml
/var/lib/jenkins/Lingoport_Data/LRM/GroupName/prep_kit/ProjectName/PREP_KIT_1/ProjectName_1_de_de.xml
/var/lib/jenkins/Lingoport_Data/LRM/GroupName/prep_kit/ProjectName/PREP_KIT_1/ProjectName_1_fr_fr.xml
Process completed successfully.

The base locale resource files are extracted and place in the appropriate localized prep kit folders. See the de_de and fr_fr folders.

> ls /var/lib/jenkins/Lingoport_Data/LRM/GroupName/prep_kit/ProjectName/PREP_KIT_1/
BaseFileSearch_de_de.xml  FilesToPrep.xml      ProjectName_1_fr_fr.xml
BaseFileSearch_fr_fr.xml  fr_fr/
de_de/                     ProjectName_1_de_de.xml

In the fr_fr folder, are all the files to be translated:

> ls /var/lib/jenkins/Lingoport_Data/LRM/GroupName/prep_kit/ProjectName/PREP_KIT_1/fr_fr/
file1_en_US.properties           
file2_en_US.properties

For missing files, the entire file will be sent. For missing keys (or modified text) only the changed keys will be sent.

How can I prevent a file or folder from being included in the prep kit?

You will need to alter your project definition file and exclude the file/folder from the project.

In Jenkins, select the project you are working on and select Configure from the menu.

AddExclude.jpg

Scroll down to the Directories section and select the Add Exclude Directory File button which will create a new entry. Add the directory or file you want to exclude. Save your project. In this example, we added the **/test/** directory.

When the project is run, the Jenkins output shows:

Creating Project Definition File: /var/lib/jenkins/Lingoport_Data/LRM/<group-name>/reports/<project-name>/ProjectDefinition.xml

If you open the Project Definition File, you should see your new addition(<exclude-dir-file>**/test/**</exclude-dir-file>) to the excludes:

<excludes>
     <exclude-dir-file>**\*_eo.properties</exclude-dir-file>
     <exclude-dir-file>**/LRMScans/**</exclude-dir-file>
     <exclude-dir-file>**/GlobalyzerScans/**</exclude-dir-file>
     <exclude-dir-file>**/sonar-project.properties</exclude-dir-file>
     <exclude-dir-file>**/test/**</exclude-dir-file>
 </excludes>

Modified text does not seem to be recognized as a change in my base resource file. What is wrong?

Modified text can only be detected if there has been a baseline created for the project. In this way, LRM can detect any modifications to file text. The Jenkins nightly Notification job for a project automatically calls the create baseline command. This command can also be called on the command line. See Create Project Baseline for more information.

When I prep a kit, what files are created?

When you prep a kit from either Jenkins or the Lingoport Dashboard and it is successful, the resulting files will be located in :

/var/lib/jenkins/Lingoport_Data/L10nStreamlining/<group-name>/projects/<project-name>/prepkits/PREP_KIT_<number>

And may contain the following files and directories.

<locales>

For each locale that needs to be translated, there is a directory named for that locale resource files for that locale. The file may be the whole resource file if no translations exist and will have FULL-FILE in the comment at the top of the file. If the file is just for changes that have occurred since the last translation, it will have just those changes in the file and CHANGES-ONLY in the comment header line.

FilesToPrep.xml

This file shows what is displayed much more nicely in the Dashboard.

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<LRMFilesToPrepTag companyName="Acme" default-locale="en_US" inContextQALicenseEn
dDate="2020-01-15" licenseEndDate="2020-01-15" licenseNumOfProjects="1000" lrm-v
ersion="5.0.28" projectID="5" projectName="Weather" timezone="GMT" topLevelD
irectory="/var/lib/jenkins/jobs/Acme.Weather/workspace/WeatherWidget-mast
er/resources">
  <locales>
    <prep-info locale="fr_CA" localeDisplayName="French (Canada)" nbFilesToTrans
late="1" nbKeysToTranslate="62" nbWordsToTranslate="82"/>
    <prep-info locale="de_DE" localeDisplayName="German (Germany)" nbFilesToTran
slate="1" nbKeysToTranslate="62" nbWordsToTranslate="82"/>
    <prep-info locale="fr_FR" localeDisplayName="French (France)" nbFilesToTrans
late="1" nbKeysToTranslate="62" nbWordsToTranslate="82"/>
    <prep-info locale="es_MX" localeDisplayName="Spanish (Mexico)" nbFilesToTran
slate="1" nbKeysToTranslate="62" nbWordsToTranslate="82"/>
  </locales>
  <files>
    <file-path file="/Weather_en_US.properties"/>
  </files>
</LRMFilesToPrepTag>
  • companyName="Acme"
  • default-locale="en_US"
  • projectName="Weather"
  • topLevelDirectory="/var/lib/jenkins/jobs/Acme.Weather/workspace/WeatherWidget-mast

er/resources"

  • For each locale to translate:
    • nbFilesToTranslate="1" - number of Files to Translate
    • nbKeysToTranslate="62" - number of Keys to Translate
    • nbWordsToTranslate="82" - number of Words to Translate
  • <files> - a list of files to translate. The files are shown in the relative path from the topLevelDirectory