Flow and Jobs

Four tabs organize the different jobs on Jenkins.

Infrastructure

Those jobs are common to all on-boarded projects and need to be set up once during installation and configuration.

checksend

The Jenkins checksend job calls the job_checksend.sh script under bin. LRM_DATA_HOME is typically set to ~jenkins/Lingoport_Data/LRM This job looks at the prep kits which have been successfully created on the local file system under <LRM_DATA_HOME>/staging/<l10nvendor> and for some reason were not sent to the translation end point (Lingotek, FTP, maybe because of some network outage). If any prep kit remains, checksend sends those prep kits to the configured end point for the project. If successful, then removes the files from the staging area.

Lingotek Infrastructure Jobs

LingotekCallback

The Jenkins LingotekCallback job calls the lingotek_webhook.sh script under bin.

The Lingotek platform calls back to a configured URL. Some callbacks are of interest, others are not. The Lingoport LRM system uses a set of callbacks to download files whose translations are complete. The files are downloaded under <LRM_DATA_HOME>/downloads/lingotek and are ready to be imported by LRM into the source code repository. (See LingotekImportKit).

The job must be a parameterized build and a number of Jenkins string parameters are necessary for this job to work properly:

• project_id
• document_id
• locale
• locale_code
• progress
• type
• complete

The Jenkins job must be triggered remotely using a specific token provided by support.

LingotekCheckCallbacks

If for some reason, the files which should have been downloaded by LingotekCallback were not downloaded, this job does its best to download the files which have been translated by Lingotek.

LingotekImportKit

The Jenkins LingotekImportKit job calls the lingotek_import.sh script under bin.

The files to be imported by Lingotek must have been downloaded (See LingotekCallback) under <LRM_DATA_HOME>/downloads/lingotek. The following steps are taken:

• Executes the checks on the translated files with regards to the project and the version of the kit
• If the checks pass, copy the translated files in a cleanly checked out project
• Uploads the files to the code repository
• Notifies the configured recipients (either of the success or of the failure of the import)
• Updates the Dashboard

This job executes on a pre-determined schedule. The typical setup would be every two hours.

FTP Infrastructure Jobs

FTPDownload

The Jenkins FTPDownload job calls the ftp_download.sh script under bin.

This job checks in all the L10nStreamlining config_l10n_vendor.properties for incoming FTP configurations. The files are downloaded, based on the configuration, under <LRM_DATA_HOME>/downloads/ftp and are ready to be imported by LRM into the source code repository. (See FSImport).

The files to be imported must conform to the following convention:

• The translated files are put in a zip file named <group_name>.<project_name>.<kit_number>.<locale>.zip, for instance Acme.projectx.13.fr.zip
• There is a root folder that contains all the translated files.
• Each file under the root folder must be in the resource bundle format: .properties files must conform to .properties format
• Each file must have kept the LRM tag at the top of the file which was set before sending the file for translation.

FTPImport

The Jenkins FTPImport job calls the ftp_import.sh script under bin.

The files to be imported by FTP must have been downloaded (See FTPDownload) under <LRM_DATA_HOME>/downloads/ftp. The following steps are taken:

• Executes the checks on the translated files with regards to the project and the version of the kit
• If the checks pass, copy the translated files in a cleanly checked out project
• Uploads the files to the code repository
• Notifies the configured recipients (either of the success or of the failure of the import)
• Updates the Dashboard

This job executes on a pre-determined schedule. The typical setup would be every two hours.

Worldserver Infrastructure Jobs

WSDownload

The Jenkins WSDownload job calls the ws_download.sh script under bin.

This job checks in all the L10nStreamlining config_l10n_vendor.properties for Worldserver incoming configurations. The files are downloaded, based on the configuration, under <LRM_DATA_HOME>/downloads/worldserver and are ready to be imported by LRM into the source code repository. (See WSImport).

The files to be imported must conform to the following convention:

• The translated files are put in a zip file where the zip file prefix matches the config_l10n_vendor.properties value of worldserver.in.file.name.prefix. For instance Acme.projectx.13.fr.zip
• There is a root folder that contains all the translated files.
• Each file under the root folder must be in the resource bundle format: .properties files must conform to .properties format
• Each file must have kept the LRM tag at the top of the file which was set before sending the file for translation.

WSImport

The Jenkins WSImport job calls the ws_import.sh script under bin.

The files to be imported by Worldserver must have been downloaded (See WSDownload) under <LRM_DATA_HOME>/downloads/worldserver. The following steps are taken:

• Executes the checks on the translated files with regards to the project and the version of the kit
• If the checks pass, copy the translated files in a cleanly checked out project
• Uploads the files to the code repository
• Notifies the configured recipients (either of the success or of the failure of the import)
• Updates the Dashboard

This job executes on a pre-determined schedule. The typical setup would be every two hours.

debug

This job is used in an ad hoc fashion to verify or try this or that functionality, for instance checking the Jenkins variables. It has not direct bearing on the functions of the system.

Automation

One Jenkins automation job is on-boarded for each repository project. The Jenkins jobs calls job_automate.sh. The Jenkins job name is composed of the group name and the project name in the form <group_name>.<project_name>.

The following steps are taken:

• The repository files are checked out anew (not update, no pull, a clean copy). Newer versions of Jenkins keep the source code under /var/lib/jenkins/workspace/<group_name>.<project_name>
• The LRM xml reports are typically generated under ~jenkins/Lingoport_Data/Dashboard/Projects/group_name.project_name/LRMScans
• The Dashboard is updated using the xml reports and relies on the above location for those xml generated reports.
• The base files are pseudo-localized and committed to the repository

The automation jobs can be triggered by a Git push or an SVN commit depending on the availability of Webhooks on the repositories. If those triggers are not set, the Nightly jobs will take care of doing something similar at least once a day, so the Dashboard and the Pseudo-localization is at most a day behind the development team commits.

To make sure the Dashboard is up to date, the Automation job can be manually triggered.

Nightlies

One Jenkins automation job is on-boarded for each repository project. The Jenkins jobs calls job_nightly.sh. The Jenkins job name is composed of the group name and the project name in the form <group_name>.<project_name>-Nightly.

In addition to what the Automation jobs do, the Nightly job also sends two emails on a configured frequency.

• A summary of the current status of the project, which is a synthesis of the Dashboard.
• A late prep kit notification if applicable: Each prep kit sent for translation has a due date, and kits which are behind are noted.

Nightlies are triggered by a cron-like scheduler, typically every night.

Prep Kits

This is the default tab. The jobs on this tab are triggered by an actual user who must have permissions.

One Jenkins automation job is on-boarded for each repository project. The Jenkins jobs calls job_prepkit.sh. The Jenkins job name is composed of the group name and the project name in the form <group_name>.<project_name>-PrepKit.

A Prep Kit job follows the following steps:

• Take the latest source code from the repository in a empty directory
• Analyze the configured resource bundles and copies the files to be translated under staging/sendkits/<l10nvendor>/<group_name>/<project_name>/PREP_KIT_x/<locale>
• Sends those files to the L10n vendor (for instance Lingotek)
• Remove the files from the staging area.
  • Note: If sending the files to the L10n vendor fails, for instance because of a network outage, the files are kept under staging. The Infractructure 'checksend' job will use those files when the corrective action (network is back) has been taken.
• Notifies the configured recipient a Prep Kit has been sent
• Determines a due date for each locale
• Updates the Dashboard

LicenseExpiration

• One Jenkins License Expiration job is on-boarded for each licensed group. The Jenkins jobs calls $LRM_HOME/bin/job_checklicense.sh 30 if 30 days is the threshold to start notification the license is about to expire. The Jenkins job name is composed of the group name and License_Expiration in the form <code <group_name>.LicenseExpiration.
• The jobs on this tab are triggered every week.
• An E-Mail Notification post-build action must specify the recipients of the license expiration.