Flow and Jobs

Four tabs organize the different jobs on Jenkins.

This illustration shows the job Blue.globalyzerswing under the Automation tab. The job is named <group name>.<project name>. In this case, the group name is "Blue" and the project name is "globalyzerswing". Lingoport recommends using this naming convention and many of the Jenkins jobs expect this.

Infrastructure

Those jobs are common to all on-boarded projects and need to be set up once during installation and configuration.

CheckSend

The Jenkins CheckSend job calls the job_checksend.sh script under <HOME>/lingoport/lrm-server-x.y/bin. LRM_DATA_HOME is typically set to <HOME>/Lingoport_Data/LRM This job looks at the prep kits which have been successfully created on the local file system under <LRM_DATA_HOME>/staging/sendkits/<l10nvendor> and for some reason were not sent to the translation end point (Lingotek, FTP, maybe because of some network outage). If any prep kit remains, CheckSend sends those prep kits to the configured end point for the project. If successful, CheckSend then removes the files from the staging area.

Lingotek Infrastructure Jobs

LingotekCallback

The Jenkins LingotekCallback job calls the lingotek_webhook.sh script under <HOME>/lingoport/lrm-server-x.y/bin.

The Lingotek platform calls back to a configured URL. Some callbacks are of interest, others are not. The Lingoport LRM system uses a set of callbacks to download files whose translations are complete. The files are downloaded under <LRM_DATA_HOME>/staging/downloads/lingotek and are ready to be imported by LRM into the source code repository. (See LingotekImportKit).

The job must be a parameterized build and a number of Jenkins string parameters are necessary for this job to work properly:

• project_id
• document_id
• locale
• locale_code
• progress
• type
• complete

The Jenkins job must be triggered remotely using a specific token provided by support.

LingotekCheckCallbacks

If for some reason, the files which should have been downloaded by LingotekCallback were not downloaded, this job does its best to download the files which have been translated by Lingotek.

LingotekImportKit

The Jenkins LingotekImportKit job calls the lingotek_import.sh script under <HOME>/lingoport/lrm-server-x.y/bin.

The files to be imported by Lingotek must have been downloaded (See LingotekCallback) under <LRM_DATA_HOME>/staging/downloads/lingotek. The following steps are taken:

• Executes the checks on the translated files with regards to the project and the version of the kit
• If the checks pass, copy the translated files in a cleanly checked out project
• Uploads the files to the code repository
• Notifies the configured recipients (either of the success or of the failure of the import)
• Updates the Dashboard

This job executes on a pre-determined schedule. The typical setup would be every two hours.

FTP Infrastructure Jobs

FTPDownload

The Jenkins FTPDownload job calls the ftp_download.sh script under <HOME>/lingoport/lrm-server-x.y/bin.

This job checks in all the L10nStreamlining config_l10n_vendor.properties for incoming FTP configurations. The zip files are downloaded, based on the configuration, under <LRM_DATA_HOME>/staging/downloads/ftp. Each zip file is then unzipped. The extracted files are renamed and moved to the appropriate directory under <LRM_DATA_HOME>/staging/importkits/ftp/<group>/<project>/TRANSLATED_KIT_<version>_<locale> where they are ready to be imported by LRM into the source code repository. (See FTPImport).

The zip files containing the files to be imported must conform to the following convention:

• The translated files are put in a zip file named <group_name>.<project_name>.<kit_number>.<locale>.zip, for instance Acme.projectx.13.fr.zip
• There is a root directory, inside the zip file, that contains all the translated files.
• Each file under the root directory must be in the resource bundle format: .properties files must conform to .properties format
• Each file must have kept the LRM tag at the top of the file which was set before sending the file for translation.

This job executes on a predetermined schedule. The typical setup would be every two hours.

FTPCheckDownload

The Jenkins FTPCheckDownload job calls the ftp_checkdownloads.sh script under <HOME>/lingoport/lrm-server-x.y/bin.

This job checks under <LRM_DATA_HOME>/staging/downloads/ftp to see if there are any outstanding zip files that have not been processed. If so, then each zip file is unzipped. The extracted files are renamed and moved to the appropriate directory under <LRM_DATA_HOME>/staging/importkits/ftp/<group>/<project>/TRANSLATED_KIT_<version>_<locale> where they are ready to be imported by LRM into the source code repository. (See FTPImport). The zip files are expected to follow the standard explained in FTPDownload.

This job executes on a predetermined schedule. The typical setup would be once a day at the most.

FTPImport

The Jenkins FTPImport job calls the ftp_import.sh script under <HOME>/lingoport/lrm-server-x.y/bin.

FTP files to be imported must have been downloaded (See FTPDownload) under <LRM_DATA_HOME>/staging/downloads/ftp. The following steps are taken:

• Executes the checks on the translated files with regards to the project and the version of the kit
• If the checks pass, copy the translated files in a cleanly checked out project
• Uploads the files to the code repository
• Notifies the configured recipients (either of the success or of the failure of the import)
• Updates the Dashboard

This job executes on a predetermined schedule. The typical setup would be to execute after the FTPDownload job has completed.

Worldserver Infrastructure Jobs

WSDownload

The Jenkins WSDownload job calls the ws_download.sh script under <HOME>/lingoport/lrm-server-x.y/bin.

This job checks in all the L10nStreamlining config_l10n_vendor.properties for incoming Worldserver configurations. The zip files are downloaded, based on the configuration, under <LRM_DATA_HOME>/staging/downloads/worldserver. Each zip file is then unzipped. The extracted files are renamed and moved to the appropriate directory under <LRM_DATA_HOME>/staging/importkits/worldserver/<group>/<project>/TRANSLATED_KIT_<version>_<locale> where they are ready to be imported by LRM into the source code repository. (See WSImport).

The zip files containing the files to be imported must conform to the following convention:

• The translated files are put in a zip file where the zip file prefix matches the config_l10n_vendor.properties value of worldserver.in.file.name.prefix. For instance Acme.projectx.13.fr.zip
• There is a root directory, inside the zip file, that contains all the translated files.
• Each file under the root directory must be in the resource bundle format: .properties files must conform to .properties format
• Each file must have kept the LRM tag at the top of the file which was set before sending the file for translation.

This job executes on a predetermined schedule. The typical setup would be every two hours.

WSCheckDownload

The Jenkins WSCheckDownload job calls the ws_checkdownloads.sh script under bin.

This job checks under <LRM_DATA_HOME>/staging/downloads/worldserver to see if there are any outstanding zip files that have not been processed. If so, then each zip file is unzipped. The extracted files are renamed and moved to the appropriate directory under <LRM_DATA_HOME>/staging/importkits/worldserver/<group>/<project>/TRANSLATED_KIT_<version>_<locale> where they are ready to be imported by LRM into the source code repository. (See WSImport). The zip files are expected to follow the standard explained in WSDownload.

This job executes on a predetermined schedule. The typical setup would be once a day at the most.

WSImport

The Jenkins WSImport job calls the ws_import.sh script under <HOME>/lingoport/lrm-server-x.y/bin.

Worldserver files that are to be imported must have been downloaded (See WSDownload) under <LRM_DATA_HOME>/staging/downloads/worldserver. The following steps are taken:

• Executes the checks on the translated files with regards to the project and the version of the kit
• If the checks pass, copy the translated files in a cleanly checked out project
• Uploads the files to the code repository
• Notifies the configured recipients (either of the success or of the failure of the import)
• Updates the Dashboard

This job executes on a predetermined schedule. The typical setup would be to execute after the WSDownload job has completed.

debug

This job is used in an ad hoc fashion to verify or try this or that functionality, for instance checking the Jenkins variables. It has not direct bearing on the functions of the system.

Automation

One Jenkins automation job is on-boarded for each repository project. The Jenkins jobs calls job_automate.sh. The Jenkins job name is composed of the group name and the project name in the form <group_name>.<project_name>.

The following steps are taken:

• The repository files are checked out anew (not update, no pull, a clean copy). Jenkins keeps the source code under <HOME>/jobs/<group_name>.<project_name>/workspace/
• The LRM XML reports are typically generated under <HOME>/Lingoport_Data/Dashboard/Projects/group_name.project_name/LRMScans
• The Lingoport Dashboard is updated using the XML reports and relies on the above location for those XML generated reports.
• The base files are pseudo-localized and committed to the repository

The automation jobs can be triggered by a Git push or an SVN commit depending on the availability of Webhooks on the repositories. If those triggers are not set, the Nightly jobs will take care of doing something similar at least once a day, so the Dashboard and the Pseudo-localization is at most a day behind the development team commits.

To make sure the Lingoport Dashboard is up to date, the Automation job can be manually triggered.

Nightlies

One Jenkins automation job is on-boarded for each repository project. The Jenkins jobs calls job_nightly.sh. The Jenkins job name is composed of the group name and the project name in the form <group_name>.<project_name>-Nightly.

In addition to what the Automation jobs do, the Nightly job also sends two emails on a configured frequency.

• A summary of the current status of the project, which is a synthesis of the Dashboard.
• A late prep kit notification if applicable: Each prep kit sent for translation has a due date, and kits which are behind are noted.

Nightlies are triggered by a cron-like scheduler in Jenkins, typically every night.

Prep Kits

This is the default tab. The jobs on this tab are triggered by an actual user who must have permissions.

One Jenkins automation job is on-boarded for each repository project. The Jenkins jobs calls job_prepkit.sh. The Jenkins job name is composed of the group name and the project name in the form <group_name>.<project_name>-PrepKit.

A Prep Kit job follows the following steps:

• Take the latest source code from the repository in a empty directory
• Analyze the configured resource bundles and copies the files to be translated under <LRM_DATA_HOME>/staging/sendkits/<l10nvendor>/<group_name>/<project_name>/PREP_KIT_x/<locale>
• Sends those files to the L10n vendor (for instance Lingotek)
• Remove the files from the staging area.
  • Note: If sending the files to the L10n vendor fails, for instance because of a network outage, the files are kept under staging. The Infractructure 'CheckSend' job will use those files when the corrective action (network is back) has been taken.
• Notifies the configured recipient a Prep Kit has been sent
• Determines a due date for each locale
• Updates the Dashboard

LicenseExpiration

• One Jenkins License Expiration job is on-boarded for each licensed group. The Jenkins jobs calls <HOME>/lingoport/lrm-server-x.y/bin/job_checklicense.sh 30 if 30 days is the threshold to start notification the license is about to expire. The Jenkins job name is composed of the group name and License_Expiration in the form <group_name>.LicenseExpiration.
• The jobs on this tab are triggered every week.
• An E-Mail Notification post-build action must specify the recipients of the license expiration.