Importing Translated Kits

To import files from translation, a number of jobs are executed on Jenkins:

• Download Translations:
  • From Lingotek side: Lingotek system calls LingotekCallback to let Jenkins know files are "translation complete";
    • Another job, LingotekCheckCallback, uses that information to retrieve the files via the Lingotek API and move them to the staging area
  • From FTP: A timed Jenkins trigger looks for all the possible FTP locations and moves the files to the staging area

• Import from staging: the system checks the files under the staging folder, imports them into the source directory at the right location, and invokes the right VCS type to push the files to the repository. This last step uses the config_vcs.properties file.
  • For Lingotek files, the Jenkins job LingotekImport is called
  • For FTP, the Jenkins job FTPImport is called.

The following sections details the locations of the files as they move through the system.

See Flow and Jobs for more information about FTP Jenkins jobs.

Lingotek

LingotekCallback

The LingotekCallback Jenkins job is invoked from the Lingotek system and uses the Build Parameters passed in the URL by Lingotek to let the Lingoport system know files are ready to be retrieved. It first creates .json files to keep track of the callback calls. From the .json files, LingotekCallback then retrieves the files and moves the translated files under <HOME>/Lingoport_Data/LRM/staging/downloads/lingotek. It then removes the .json files.

If any problem occurs during the retrieval of the files (network outage), the .json files are then used by LingotekCheckCallbacks later on.

LingotekCheckCallbacks

This job looks at the .json files and for each one retrieves the files from Lingotek to <HOME>/Lingoport_Data/LRM/staging/downloads/lingotek/. It then moves the files from <HOME>/Lingoport_Data/LRM/staging/downloads/lingotek/ to <HOME>/Lingoport_Data/LRM/staging/importkits/lingotek/<group_name>/<project_name>/TRANSLATED_KIT_<kit_version>_<locale>

The files in <HOME>/Lingoport_Data/LRM/staging/importkits can now be used to import into the source code.

LingotekImportKit

This Jenkins job uses the files under <HOME>/Lingoport_Data/LRM/staging/importkits/lingotek/<group_name>/<project_name>/TRANSLATED_KIT_<kit_version>_<locale> to import into the source code. If successful, an email notifies the recipient and the proper VCS is invoked to push the files to the repository.

Finally, for tracking purposes, a copy of the translated files retrieved from Lingotek is kept under <HOME>/L10nStreamlining/<group_name>/projects/<project_name>/importkits/TRANSLATED_KIT_<kit_version>_<locale>

FTP

FTPDownload

For FTP, the files are read from the ftp.in.location.path configuration in config_l10n_vendor.properties. The files found in the FTP endpoint are moved to <HOME>/Lingoport_Data/LRM/staging/downloads/ftp/.

They must conform to the zip structure and naming convention.

At that point, the files are ready to be imported. The Jenkins job FTPDownload is in charge of that job.

FTPImport

The first step is to collect the files from the source directly, after a clean checkout/clone from the repository. If all is copacetic, the files to be send for translation are moved under the core LRM data directory, LRM_DATA_HOME, which is typically set to <HOME>/Lingoport_Data/LRM.

For instance, if this is the second prep kit, the base files would be moved under:

<HOME>/Lingoport_Data/LRM/<group_name>
├── logs
├── prep_kit
│   └── <project>
│       └── PREP_KIT_2

WorldServer

WSDownload

For Worldserver, the files are read from the worldserver.in.location.path configuration in config_l10n_vendor.properties. The files found in the FTP endpoint are moved to <HOME>/Lingoport_Data/LRM/staging/downloads/worldserver/

They must conform to the zip structure and naming convention.

At that point, the files are ready to be imported. The Jenkins job WSDownload is in charge of that job.

Moving the Prep Kit under staging

If that stage is successful, the files are then getting ready to be sent and are placed under the <HOME>/Lingoport_Data/LRM/staging/sendkits/lingotek or <HOME>/Lingoport_Data/LRM/staging/sendkits/ftp depending on the project configuration:

<HOME>/Lingoport_Data/LRM/staging
├── ...
├── sendkits
│   └── lingotek
│       ├── <group_name>
│       │   └── <project_name>
│               └── PREP_KIT_2
│                   ├── BaseFileSearch_fr.xml
│                   ├── FilesToPrep.xml
│                   ├── fr
│                   │   ├── file1_en.properties
│                   │   └── file2_en.properties
│                   ├── <project_name>_1_fr.xml
│                   └── IncompletePrepKit.xml

Sending the Files under staging

If this has been successful, the next step is to send the files to Lingotek or FTP, depending on the configuration. This may fail due to network problems, server saturation, or other causes. Until the files have successfully been sent, they are kept under staging.

If the files were not sent, check_send will later on pick up here in the process

Keeping Track under L10nStreamlining

L10nStreamlining helps keep track of what was sent and received. The last stage of the Prep Kit process is to move the files which were under

• <HOME>/Lingoport_Data/LRM/staging/sendkits

over to

• <HOME>/Lingoport_Data/L10nStreamlining/<group_name>/projects<project_name>/prepkits/PREP_KIT_X.