Difference between revisions of "Deployment Scenarios"

From Lingoport Wiki
Jump to: navigation, search
(A typical workflow=)
(Lingoport Dashboard)
(36 intermediate revisions by 2 users not shown)
Line 4: Line 4:
 
Setting up the [[Terms_and_Definitions#continuousglobalizationsystem|Lingoport Continuous Globalization System]] at a customer site can be done in many ways, as shown in the different deployment scenarios. The following is meant to show some of the flow and the necessary access to the system.
 
Setting up the [[Terms_and_Definitions#continuousglobalizationsystem|Lingoport Continuous Globalization System]] at a customer site can be done in many ways, as shown in the different deployment scenarios. The following is meant to show some of the flow and the necessary access to the system.
   
  +
[[File:Deployment - Wiki.gif|500px]]
   
  +
The green box is the '''Continuous Globalization System'''. It is a Linux (CentOS or RedHat) operating system that has the following installed:
*The Customer's systems are shown in Light Blue; <b>CentOS</b> for the main system where [[Terms_and_Definitions#Jenkins |Jenkins]] and [[Terms_and_Definitions#Dashboard|Dashboard]] reside.
 
  +
* Jenkins
*Access from outside the Customer's system are shown in Light Green.
 
 
[[Image:Deployment and Security.gif]]
 
 
The large blue box is the Lingoport Server. It is a Linux (CentOS or RedHat) operating system that has the following installed:
 
* Jenkins (running on port 8080)
 
 
* Java
 
* Java
 
* MySQL
 
* MySQL
 
* Globalyzer clients
 
* Globalyzer clients
* Lingoport Dashboard Server (running on port 9000) and Database
+
* Lingoport Dashboard Server and Database
 
* Lingoport Dashboard Client
 
* Lingoport Dashboard Client
 
* LRM Server and Database
 
* LRM Server and Database
Line 21: Line 17:
   
 
The Globalzyer Server is either a hosted Lingoport server or another onsite server.
 
The Globalzyer Server is either a hosted Lingoport server or another onsite server.
 
The optional items shown:
 
* updates.jenkins.org: The Lingoport Server should be able to download updates to Jenkins.
 
* SSH: Allowing Lingoport to log in to the customer Lingoport Server improves our ability to support the products.
 
* updates.sonarsource.org: The Lingoport Dashboard uses SonarQube as its framework and needs the ability to download updates and fixes.
 
   
 
==A typical workflow==
 
==A typical workflow==
# Code is committed to a central source code repository (Git, Subversion, etc) by developers
+
# Code is committed to a central source code repository (Git, Subversion, etc) by developers.
# Jenkins tools pull the source code from the code repository and scan using Rule Sets stored on the Globalyzer Server or analyze the resource files
+
# Jenkins tools pull the source code from the code repository and scan using Rule Sets stored on the Globalyzer Server or analyze the resource files.
 
# Jenkins tools determine if the resource files need to be sent to, or imported from, the translation vendor. This may be done via an FTP connection.
 
# Jenkins tools determine if the resource files need to be sent to, or imported from, the translation vendor. This may be done via an FTP connection.
 
# Emails are sent to designated recipients about the status of the translation upload or import.
 
# Emails are sent to designated recipients about the status of the translation upload or import.
# Lingoport Dashboard updates the status using the results of the Globalyzer scans and LRM resource status.
+
# Lingoport Dashboard updates the status using the results of the Globalyzer scans and LRM resource reports.
   
= External Access and Ports =
+
== External Access and Ports ==
  +
The Lingoport Server:
* Ability to install/update programs via 'yum'.
 
  +
* [[Terms_and_Definitions#Jenkins|Jenkins]] is accessed via port 8080 within Customer's firewall for Linux system
 
  +
* Must have the ability to install/update programs via 'yum'.
* [[Terms_and_Definitions#LingoportDashboard|Lingoport Dashboard]] is accessed via port 9000 within Customer's firewall for Linux system
 
  +
* [[Terms_and_Definitions#Jenkins|Jenkins]] is accessed (via port 8080 if installed prior to May 2018) within Customer's firewall for Linux system
  +
* [[Terms_and_Definitions#LingoportDashboard|Lingoport Dashboard]] is accessed (via port 9000 if installed prior to May 2018) within Customer's firewall for Linux system
 
* The [[Terms_and_Definitions#translationvendor|Translation Vendor]] may be accessed by different means, for instance port 21 for FTP or port 22 for SFTP.
 
* The [[Terms_and_Definitions#translationvendor|Translation Vendor]] may be accessed by different means, for instance port 21 for FTP or port 22 for SFTP.
 
* Download does not have to be directly to the target machine. It can be downloaded by another machine and then transferred.
 
* Download does not have to be directly to the target machine. It can be downloaded by another machine and then transferred.
  +
* Lingoport requests SSH access to the Lingoport Server. This allows Lingoport to best debug and troubleshoot problems. If this is not possible, Lingoport can guide a customer through the setup using a videoconferencing program such as WebEx or GoToMeeting.
* Access for the Lingoport (or internal) installation team can be done in many ways, such as SSH for Telnet or Putty.
 
* Linux system must be able to access https://globalyzer.com.
+
* The Lingoport Server must be able to access https://globalyzer.com, especially if using Lingoport's hosted Globalyzer Server.
   
 
==Optional Access==
 
==Optional Access==
Line 52: Line 45:
   
 
Furthermore, email notification emails require an SMTP account. The SMTP server access may be internal to your Company or external depending on your email services (e.g. smtp.gmail.com for Google).
 
Furthermore, email notification emails require an SMTP account. The SMTP server access may be internal to your Company or external depending on your email services (e.g. smtp.gmail.com for Google).
 
   
 
= Installation Components =
 
= Installation Components =
 
== Globalyzer==
 
== Globalyzer==
Scan code and detect internationalization (i18n) issues
+
Globalyzer scans code and detect [[Terms_and_Definitions#internationalization|internationalization (i18n)]] issues.
   
 
===Globalyzer Server===
 
===Globalyzer Server===
  +
The Globalyzer Server stores regular expression (regex) scanning rules called [[Terms_and_Definitions#ruleset|Rule Sets]] that are used to identify issues in source code.
Stores (regex) scanning rules.
 
   
* Stores regex pattern based 'rule sets' used to detect i18n issues and filter out false positives and Globalyzer accounts.
+
* Stores regex pattern based 'rule sets' used to detect i18n issues and filter out [[Terms_and_Definitions#falsepositive|false positives]] and Globalyzer accounts.
* Does not store any code or customer information.
+
* The Globalyzer Server does not store any code or customer information.
   
 
===Globalyzer Clients===
 
===Globalyzer Clients===
  +
The Globalyzer clients display i18n issues found by the Globalyzer [[Terms_and_Definitions#scan|scan]].
Displays i18n issues.
 
   
* Connects to server and logs in. Downloads rule set from server. Uses rule set to scan code (no code is sent to the server!). May push rule set changes.
+
* The Globalyzer clients connect to the Globalyzer Server. The clients log in and download [[Terms_and_Definitions#ruleset|rule sets]] from the Globalylzer server to scan code. No code is sent to the Globalyzer server, although the Globalyzer clients may push rule set modifications.
* Run on developer machines
+
* The Globalyzer clients run on developer machines.
* Run on Linux system (light-blue box at bottom of graphic) - results displayed on Lingoport Dashboard (see data flow)
+
* The Globalyzer clients also run on the Lingoport Server (light-blue box at bottom of graphic) and the results are displayed on the [[Terms_and_Definitions#LingoportDashboard|Lingoport Dashboard]] (see data flow)
   
==Resource Manager==
+
==Resource Manager (LRM) ==
Manages translation resource files.
+
The Lingoport Resource Manager (LRM) manages translation [[Terms_and_Definitions#resourcefiles|resource files]].
* Detects issues in resource files (duplicate or missing keys, parameter mismatch in text for different languages, etc.)
+
* LRM detects problems in resource files (duplicate or missing keys, parameter mismatch in text for different languages, etc.)
* Detects changes to Chinese resource files. Then sends relevant changes out to the translation vendor for translation into other languages.
+
* LRM also detects changes to resource files and sends relevant changes out to the [[Terms_and_Definitions#translationvendor|translation vendor]] for translation into other languages.
* Automatically retrieves translations from translations vendors and checks those changes into source control
+
* LRM automatically retrieves translations from translations vendors and checks those changes into the code source control repository.
* Run on Linux system (light-blue box at bottom of graphic)
+
* LRM runs on the Lingoport Server (light-blue box at bottom of graphic).
   
 
==Lingoport Dashboard==
 
==Lingoport Dashboard==
Displays internationalization and localization status.
+
The Lingoport Dashboard, a web application, displays [[Terms_and_Definitions#internationalization|internationalization]] and [[Terms_and_Definitions#localization|localization]] status using Globalyzer and LRM reports.
   
* Displays overview of Globalyzer and Resource Manager status
+
* Displays overview and detail views of Globalyzer scan results and Resource Manager translation status.
* Users may view Globalyzer/Resource Manager issues in context within source code
+
* Users can view Globalyzer and Resource Manager issues in context within the source code.
* Run on linux system (light-blue box at bottom of graphic)
+
* Lingoport Dashboard runs on the Lingoport Server (light-blue box at bottom of graphic).
  +
  +
===Lingoport Dashboard Client===
  +
The Lingoport Dashboard client runs the scripts which analyzes the projects and creates the XML files that get pushed to the server.
  +
  +
===Lingoport Dashboard Server===
  +
The Lingoport Dashboard server displays the results of the Globalyzer Scans and LRM reports in a web application.
   
 
==Jenkins (not a Lingoport program)==
 
==Jenkins (not a Lingoport program)==
Used to automate running of Globalyzer Client, Resource Manager and updating of the Lingoport Dashboard
+
Jenkins is used to automate running of Globalyzer Client, Resource Manager and updating of the Lingoport Dashboard.
  +
* Several tools created by Lingoport are used to run automation and analysis scripts from the Jenkins interface for the Lingoport Dashboard.
* Run on Linux system (light-blue box at bottom of graphic)
 
  +
* Jenkins runs on the Lingoport Server (light-blue box at bottom of graphic)
   
 
=Data Flow=
 
=Data Flow=
 
==Lingoport Access==
 
It's most convenient if Lingoport can have SSH access to the Linux system where Lingoport suite is configured.
 
Otherwise, a Lingoport employee can guide an customer's employee through the setup using a videoconferencing program such as WebEx or GoToMeeting.
 
   
 
== Globalyzer Process ==
 
== Globalyzer Process ==
 
[[File:Lingoport_Suite_in_Enterprise.png|700px]]
 
[[File:Lingoport_Suite_in_Enterprise.png|700px]]
  +
* The internationalization team creates and configures [[Terms_and_Definitions#rulesets| rule sets]] on the [[Terms_and_Definitions#GlobalyzerServer|Globalyzer Server]]. Rule sets are based on the coding languages used.
 
  +
* Developers can use the [[Terms_and_Definitions#GlobalyzerLite|Globalyzer Lite]] client with the rule sets to validate their code locally before committing to a repository.
Rule sets (regex data) are transferred between Globalyzer Server and Globalyzer Clients; This requires username/password based login.
 
  +
* A Jenkins tool scans the code using the rule sets, creates XML reports, and pushes the results to the Lingoport Dashboard Server using the Lingoport Dashboard Client.
* Server at Globalyzer.com
 
* Clients are run on developer machines.
 
* Clients are run on Lingoport Suite Linux system.
 
** Clients generate XML reports that will be read by the Lingoport Dashboard.
 
   
 
==Resource Manager Process ==
 
==Resource Manager Process ==
Sends to-be-translated resource files updates to translation vendor, typically over SFTP. Resource file updates received from translation vendor, also typically over SFTP. Updates are checked for consistency (various in-depth checks), and then committed to source control -- if the checks pass.
 
   
[[File:Lingoport_Resource_Manager_Prep-Kit_Process_Diagram.png|700px]]
+
<!-- [[File:Lingoport_Resource_Manager_Prep-Kit_Process_Diagram.png|700px]] -->
   
* Emails are sent to a list of email contacts defined in a configuration file.
 
** Notifications for sent / received resource files
 
** Error notifications
 
** Translation status weekly email
 
* Stores data in MySQL database
 
* Run on Linux system (light-blue box at bottom of graphic)
 
* Generates XML reports that will be read by the Lingoport Dashboard.
 
 
==Lingoport Dashboard==
 
Internal within customer network. XML Reports generated by Globalyzer and Resource Manager are uploaded via the Dashboard Client to the Dashboard Server. Dashboard Client sends code information to Dashboard Server.
 
 
* Resource Manager and Globalyzer are run on a server internal to Customer's network. Each generates an XML report.
 
* Dashboard Client reads both source code and these XML reports.
 
* Dashboard Client processes this data, and sends it to the Dashboard Server.
 
* Data sent over HTTP/HTTPS.
 
* Most often, the Dashboard Client and Dashboard Server are hosted on the same machine (light-blue box at bottom of graphic), so network communication is internal to this machine.
 
* Requires a either a username/password or a user token, which will be stored in configuration files.
 
* Stores data in MySQL database.
 
* Dashboard Server is a web application, hosted on port 9000 by default.
 
 
==Jenkins (not a Lingoport program)==
 
Used to automate running of Globalyzer Client, Resource Manager and updating of the Lingoport Dashboard
 
   
  +
*[[Terms_and_Definitions#ResourceFiles|Resource Files]] and Bundles are pulled from the Source Control Repository to the [[Terms_and_Definitions#LRMServer|LRM Server]].
* Various security options available, username/password is most common. LDAP is another option.
 
  +
* LRM determines if there are problems with the resource files or if there are new keys to be translated.
* Run on Linux system (light-blue box at bottom of graphic)
 
  +
* If there are keys to be translated, a [[Terms_and_Definitions#prepkit|Prep Kit]] is created using MySQL to track the translation.
* Web application, hosted on port 8080 by default.
 
  +
* The prep kit files (one per translation locale) are sent to the translation vendor via SFTP and success or failure emails are sent to designated recipients.
  +
* When the prep kits have been translated, LRM downloads and imports the files and sends status emails.
  +
* LRM pushes the new resource files into the source code repository.
  +
* The Lingoport Dashboard updates its status using the LRM reports.
   
 
= Additional Deployments Scenarios =
 
= Additional Deployments Scenarios =

Revision as of 22:02, 23 July 2018

The Lingoport Suite is composed of Globalyzer, LRM (Resource Manager) and the Lingoport Dashboard. A number of other products are used with the Suite. You may need some of the components for a set of users and other components for Continuous Globalization.

Typical Deployment and Ports

Setting up the Lingoport Continuous Globalization System at a customer site can be done in many ways, as shown in the different deployment scenarios. The following is meant to show some of the flow and the necessary access to the system.

Deployment - Wiki.gif

The green box is the Continuous Globalization System. It is a Linux (CentOS or RedHat) operating system that has the following installed:

  • Jenkins
  • Java
  • MySQL
  • Globalyzer clients
  • Lingoport Dashboard Server and Database
  • Lingoport Dashboard Client
  • LRM Server and Database
  • SMTP mail server account

The Globalzyer Server is either a hosted Lingoport server or another onsite server.

A typical workflow

  1. Code is committed to a central source code repository (Git, Subversion, etc) by developers.
  2. Jenkins tools pull the source code from the code repository and scan using Rule Sets stored on the Globalyzer Server or analyze the resource files.
  3. Jenkins tools determine if the resource files need to be sent to, or imported from, the translation vendor. This may be done via an FTP connection.
  4. Emails are sent to designated recipients about the status of the translation upload or import.
  5. Lingoport Dashboard updates the status using the results of the Globalyzer scans and LRM resource reports.

External Access and Ports

The Lingoport Server:

  • Must have the ability to install/update programs via 'yum'.
  • Jenkins is accessed (via port 8080 if installed prior to May 2018) within Customer's firewall for Linux system
  • Lingoport Dashboard is accessed (via port 9000 if installed prior to May 2018) within Customer's firewall for Linux system
  • The Translation Vendor may be accessed by different means, for instance port 21 for FTP or port 22 for SFTP.
  • Download does not have to be directly to the target machine. It can be downloaded by another machine and then transferred.
  • Lingoport requests SSH access to the Lingoport Server. This allows Lingoport to best debug and troubleshoot problems. If this is not possible, Lingoport can guide a customer through the setup using a videoconferencing program such as WebEx or GoToMeeting.
  • The Lingoport Server must be able to access https://globalyzer.com, especially if using Lingoport's hosted Globalyzer Server.

Optional Access

You may want to update either Jenkins or the underlying Dashboard platform to the latest version. If so, you may want to update the continuous globalization system directly or to download the files on one system and copy them over to the continuous globalization system. The URL for updates are:

Furthermore, email notification emails require an SMTP account. The SMTP server access may be internal to your Company or external depending on your email services (e.g. smtp.gmail.com for Google).

Installation Components

Globalyzer

Globalyzer scans code and detect internationalization (i18n) issues.

Globalyzer Server

The Globalyzer Server stores regular expression (regex) scanning rules called Rule Sets that are used to identify issues in source code.

  • Stores regex pattern based 'rule sets' used to detect i18n issues and filter out false positives and Globalyzer accounts.
  • The Globalyzer Server does not store any code or customer information.

Globalyzer Clients

The Globalyzer clients display i18n issues found by the Globalyzer scan.

  • The Globalyzer clients connect to the Globalyzer Server. The clients log in and download rule sets from the Globalylzer server to scan code. No code is sent to the Globalyzer server, although the Globalyzer clients may push rule set modifications.
  • The Globalyzer clients run on developer machines.
  • The Globalyzer clients also run on the Lingoport Server (light-blue box at bottom of graphic) and the results are displayed on the Lingoport Dashboard (see data flow)

Resource Manager (LRM)

The Lingoport Resource Manager (LRM) manages translation resource files.

  • LRM detects problems in resource files (duplicate or missing keys, parameter mismatch in text for different languages, etc.)
  • LRM also detects changes to resource files and sends relevant changes out to the translation vendor for translation into other languages.
  • LRM automatically retrieves translations from translations vendors and checks those changes into the code source control repository.
  • LRM runs on the Lingoport Server (light-blue box at bottom of graphic).

Lingoport Dashboard

The Lingoport Dashboard, a web application, displays internationalization and localization status using Globalyzer and LRM reports.

  • Displays overview and detail views of Globalyzer scan results and Resource Manager translation status.
  • Users can view Globalyzer and Resource Manager issues in context within the source code.
  • Lingoport Dashboard runs on the Lingoport Server (light-blue box at bottom of graphic).

Lingoport Dashboard Client

The Lingoport Dashboard client runs the scripts which analyzes the projects and creates the XML files that get pushed to the server.

Lingoport Dashboard Server

The Lingoport Dashboard server displays the results of the Globalyzer Scans and LRM reports in a web application.

Jenkins (not a Lingoport program)

Jenkins is used to automate running of Globalyzer Client, Resource Manager and updating of the Lingoport Dashboard.

  • Several tools created by Lingoport are used to run automation and analysis scripts from the Jenkins interface for the Lingoport Dashboard.
  • Jenkins runs on the Lingoport Server (light-blue box at bottom of graphic)

Data Flow

Globalyzer Process

Lingoport Suite in Enterprise.png

  • The internationalization team creates and configures rule sets on the Globalyzer Server. Rule sets are based on the coding languages used.
  • Developers can use the Globalyzer Lite client with the rule sets to validate their code locally before committing to a repository.
  • A Jenkins tool scans the code using the rule sets, creates XML reports, and pushes the results to the Lingoport Dashboard Server using the Lingoport Dashboard Client.

Resource Manager Process

  • Resource Files and Bundles are pulled from the Source Control Repository to the LRM Server.
  • LRM determines if there are problems with the resource files or if there are new keys to be translated.
  • If there are keys to be translated, a Prep Kit is created using MySQL to track the translation.
  • The prep kit files (one per translation locale) are sent to the translation vendor via SFTP and success or failure emails are sent to designated recipients.
  • When the prep kits have been translated, LRM downloads and imports the files and sends status emails.
  • LRM pushes the new resource files into the source code repository.
  • The Lingoport Dashboard updates its status using the LRM reports.

Additional Deployments Scenarios

In the Additional Deployments section, we illustrate some possible deployments:

  • Team Members
    • i18n specialists
    • Developers
  • Other Continuous Globalization Deployments