Difference between revisions of "Localyzer Machine Translation through a Proxy Server"

From Lingoport Wiki
Jump to: navigation, search
(Rational)
 
(42 intermediate revisions by the same user not shown)
Line 1: Line 1:
= Rational =
+
= Rationale =
   
A proxy Server may be used for Localyzer ''AWS Machine Translation'' if necessary for IT/Network/Security requirements.
+
A proxy Server may be used for Localyzer Machine Translation through '''Amazon Translate''' if necessary for IT/Network/Security requirements. Only Amazon Translate is supported as of 2020-11-05.
  +
  +
This is done by setting the ''''https_proxy'''' environment variable. Another variable, 'http_proxy', is available for http connections, but will not be needed for use with Amazon Translate.
  +
  +
= Overview =
  +
  +
== Definition ==
  +
  +
Definition of a proxy server per [https://en.wikipedia.org/wiki/Proxy_server#Forward_proxies wikipedia]:
  +
  +
In computer networking, a proxy server is a server application or appliance that acts as an intermediary for requests from clients seeking resources from servers that provide those resources. A proxy server thus functions on behalf of the client when requesting service.
  +
  +
== Purpose ==
  +
  +
A proxy may be used when direct access from a Lingoport System to Amazon Translate is not supported by the network configuration, but IT permits external access through a proxy.
  +
  +
= Requirements =
  +
  +
A forward http proxy server must be made available by your IT group. It must be possible to connect to this proxy server from the Lingoport Suite System.
  +
  +
You will need to know:
  +
# The URL and/or IP address of the proxy you will use
  +
# The port you will connect to the proxy on
  +
  +
The proxy will need to support:
  +
# Passthrough connections to https://translate.<region>.amazonaws.com, where <region> may be us-east-1, us-east-2, us-west-1, us-west-2, or any other region as supported by Amazon Translate. E.g. https://translate.us-east-1.amazonaws.com
  +
  +
For validation purposes, it is recommended that the proxy also support connections to an external site that may report your IP address, such as but not limited to https://canhazip.com .
   
This is done by setting the 'https_proxy' environment variable. Another variable, 'http_proxy', is available for http connections, but will not be needed for use with Amazon Translate.
 
   
 
= Proxy Validation =
 
= Proxy Validation =
   
It is recommended that you validate you have access to your proxy from your system, prior to enabling it.
+
It is recommended that you validate you have access to the proxy you'll use, from your system, prior to enabling it.
   
 
Test steps (example proxy used is http://masnes.testproxy.internal.lingoport.io:3128):
 
Test steps (example proxy used is http://masnes.testproxy.internal.lingoport.io:3128):
  +
* SSH to the Lingoport Suite System
* ssh to your system
 
* install 'telnet', if it is not already present
+
* Install 'telnet', if it is not already present <code>sudo yum install telnet</code>
  +
* Validate that you can connect to your proxy server, while keeping proxy settings clear - for now:
:: <code>sudo yum install telnet</code>
 
  +
:: <code>http_proxy="" https_proxy="" telnet <proxyurl-no-http/s> <proxyport></code>
* validate that you can connect to your proxy server, while keeping proxy settings clear - for now:
 
  +
:: Example:
:: <code>http_proxy="" https_proxy="" telnet <proxyurl-no-http/s> <proxyport><code>
 
  +
::: <code>http_proxy="" https_proxy="" telnet masnes.testproxy.internal.lingoport.io 3128</code>
: Example:
 
  +
:: Result:
:: <code>http_proxy="" https_proxy="" telnet masnes.testproxy.internal.lingoport.io 3128</code>
 
  +
::: <code>Trying 172.31.81.146...</code>
: Result
 
  +
::: <code>Connected to masnes.testproxy.internal.lingoport.io.</code>
:: Trying 172.31.81.146...
 
  +
::: <code>Escape character is '^]'.</code>
:: Connected to masnes.testproxy.internal.lingoport.io.
 
  +
::: '''Note:'''After connection is established, <code>Enter</code> will quit and return you to your terminal. If connection takes a while, use <code>Ctrl-C</code> instead.
:: Escape character is '^]'.
 
  +
:: If the connection '''takes more than 5 seconds''', you may '''not''' be able to connect to the proxy. '''Please check your security groups / firewall / networking settings before continuing!'''
:: Note: <code>Enter</code> will quit and return you to your terminal.
 
  +
* Next, as a comparison point, try curling to an external site that will return your ip address, without using the proxy settings yet.
: If the connection takes more than 5 seconds, you may not be able to connect to the proxy, please check your networking connection settings before continuing.
 
* Next, as a comparison point, try curling + to an external site that will return the found ip address, without using the proxy settings yet.
 
 
:: Here, we use 'https://canhazip.com' as the external site. We set a timeout in case the connection is blocked.
 
:: Here, we use 'https://canhazip.com' as the external site. We set a timeout in case the connection is blocked.
 
: Command
 
: Command
Line 29: Line 54:
 
: Expected Result
 
: Expected Result
 
:: <your systems ip>, something like '34.195.141.162'
 
:: <your systems ip>, something like '34.195.141.162'
: Alternate Expected Result - You'll see this if you are blocked from external connections by your firewall (possibly what you're using the proxy to get avoid):
+
: Alternate Expected Result - You'll see this if you are blocked from external connections by your firewall (possibly what you're using the proxy to avoid!):
 
:: NOTE ...Connection timed out after 5 seconds.
 
:: NOTE ...Connection timed out after 5 seconds.
* Next, try using the proxy with curl + an external site that will return the found ip address.
+
* Next, try using the proxy with curl + an external site that will return your ip address.
 
: Command
 
: Command
 
:: <code>https_proxy="http://<proxyurl>:<proxyport>" curl https://canhazip.com --silent --connect-timeout 5 || echo "FAILED: With proxy, connection timed out after 5 seconds."</code>
 
:: <code>https_proxy="http://<proxyurl>:<proxyport>" curl https://canhazip.com --silent --connect-timeout 5 || echo "FAILED: With proxy, connection timed out after 5 seconds."</code>
Line 37: Line 62:
 
:: <code>https_proxy="http://masnes.testproxy.internal.lingoport.io:3128" curl https://canhazip.com --silent --connect-timeout 5 || echo "FAILED: With proxy, connection timed out after 5 seconds."</code>
 
:: <code>https_proxy="http://masnes.testproxy.internal.lingoport.io:3128" curl https://canhazip.com --silent --connect-timeout 5 || echo "FAILED: With proxy, connection timed out after 5 seconds."</code>
 
: Expected Result
 
: Expected Result
:: <proxy ip>, something different from the above. For the above example it was '54.205.127.98'. The connection should ''not'' fail.
+
:: <proxy ip>, something different from the above. For the above example it was '54.205.127.98', which was different from the previous '34.195.141.162'. This time, the connection must '''not''' fail.
   
 
If you're able to connect to the proxy via telnet, you're able to use the proxy to get your remote ip from https://canhazip.com, and if the result you get from https://canhazip.com via the proxy is different than without the proxy, then you are ready to proceed.
 
If you're able to connect to the proxy via telnet, you're able to use the proxy to get your remote ip from https://canhazip.com, and if the result you get from https://canhazip.com via the proxy is different than without the proxy, then you are ready to proceed.
   
= Defining the Proxy for use with LRM AWS Machine Translate =
+
= Defining the Proxy for use with Localyzer Amazon Translate =
   
To set a proxy server, define the environment variable "https_proxy" prior to running LRM with AWS Machine Translation. Through Jenkins, the easiest way to do so is:
+
To set a proxy server, define the environment variable "https_proxy" prior to running Localyzer with Amazon Translate. Through Jenkins, the easiest way to do so is:
   
 
: Manage Jenkins -> (Goes to a new page)
 
: Manage Jenkins -> (Goes to a new page)
Line 56: Line 81:
 
https_proxy
 
https_proxy
 
http://masnes.testproxy.internal.lingoport.io:3128
 
http://masnes.testproxy.internal.lingoport.io:3128
  +
  +
[[File:Jenkins-config-env-https proxy.png]]
  +
  +
Click either 'Save' or 'Apply' once you are finished.
   
 
= Validating the Proxy through Jenkins =
 
= Validating the Proxy through Jenkins =
Line 61: Line 90:
 
You can add a step to your Jenkins Automation build(s) to confirm that the proxy is used. To do so:
 
You can add a step to your Jenkins Automation build(s) to confirm that the proxy is used. To do so:
   
: Start at the Jenkins HomePage
+
* Start at the Jenkins HomePage
: Select the build you would like to test with. For example 'MyCompany.MyProject' -> (Goes to a new page)
+
* Select the build you would like to test with. For example 'MyCompany.MyProject' -> (Goes to a new page)
: Go to Configure on the left -> (Goes to a new page)
+
* Go to Configure on the left -> (Goes to a new page)
  +
* Scroll to the bottom of the Configure page. You should see an 'Add build step' button just '''above''' 'Post-build actions'
:
 
  +
* Add build step -> Execute shell -> (An empty 'Execute shell' box should appear)
  +
[[File:Jenkins-add-build-step-execute-shell.png]]
  +
* In the box, add the following content:
  +
<pre>
  +
set +e # curl may fail without the proxy, depending on network configuration.
  +
set +x # optional setting. Don't show commands run, so output is clearer.
  +
echo
  +
echo "Validating Proxy is Used:"
  +
echo
  +
echo "Attempting connection without proxy. May timeout if network blocks traffic without proxy."
  +
echo "Otherwise, should return the external ip of the system."
  +
echo -n "IP returned: "
  +
https_proxy="" curl https://canhazip.com --silent --connect-timeout 5 || echo "NOTE: Without proxy, connection timed out after 5 seconds."
  +
echo
  +
echo "Attempting connection with proxy. Proxy is valid if connection succeeds."
  +
echo "Should return the external ip of the proxy server."
  +
echo " details: https_proxy set to: $https_proxy"
  +
echo -n "IP returned: "
  +
curl "https://canhazip.com" --silent --connect-timeout 5 || echo "FAILED: With the proxy set, connection timed out after 5 seconds."
  +
echo
  +
echo
  +
</pre>
  +
* After you have filled it out, drag and drop this box so that it is moved higher, above any other boxes such as 'Update Dashboard' or 'L10n Vendor'. This way it runs first.
  +
[[File:Jenkins-proxy-build-step-execute-shell-above-run-lrm.png]]
  +
* Save the build. Then run it. Navigate to the console output of the build. You should see a section that shows:
  +
<pre>
  +
Validating Proxy is Used:
  +
  +
Attempting connection without proxy. May timeout if network blocks traffic without proxy.
  +
Otherwise, should return the external ip of the system.
  +
IP returned: 34.195.141.162
  +
  +
Attempting connection with proxy. Proxy is valid if connection succeeds.
  +
Should return the external ip of the proxy server.
  +
details: https_proxy set to: http://masnes.testproxy.internal.lingoport.io:3128
  +
IP returned: 54.205.127.98
  +
</pre>
  +
  +
Like so:
  +
  +
[[File:Jenkins-proxy-build-output-validate-proxy.png]]
  +
  +
  +
You are now using an https proxy!
  +
  +
Please see [[Machine Translation]] for details of Localyzer Machine Translation. Proxy configurations have been validated with '''Amazon Translate'''. It will take effect automatically as long as the https_proxy environment variable is set.

Latest revision as of 21:13, 5 November 2020

Rationale

A proxy Server may be used for Localyzer Machine Translation through Amazon Translate if necessary for IT/Network/Security requirements. Only Amazon Translate is supported as of 2020-11-05.

This is done by setting the 'https_proxy' environment variable. Another variable, 'http_proxy', is available for http connections, but will not be needed for use with Amazon Translate.

Overview

Definition

Definition of a proxy server per wikipedia:

In computer networking, a proxy server is a server application or appliance that acts as an intermediary for requests from clients seeking resources from servers that provide those resources. A proxy server thus functions on behalf of the client when requesting service.

Purpose

A proxy may be used when direct access from a Lingoport System to Amazon Translate is not supported by the network configuration, but IT permits external access through a proxy.

Requirements

A forward http proxy server must be made available by your IT group. It must be possible to connect to this proxy server from the Lingoport Suite System.

You will need to know:

  1. The URL and/or IP address of the proxy you will use
  2. The port you will connect to the proxy on

The proxy will need to support:

  1. Passthrough connections to https://translate.<region>.amazonaws.com, where <region> may be us-east-1, us-east-2, us-west-1, us-west-2, or any other region as supported by Amazon Translate. E.g. https://translate.us-east-1.amazonaws.com

For validation purposes, it is recommended that the proxy also support connections to an external site that may report your IP address, such as but not limited to https://canhazip.com .


Proxy Validation

It is recommended that you validate you have access to the proxy you'll use, from your system, prior to enabling it.

Test steps (example proxy used is http://masnes.testproxy.internal.lingoport.io:3128):

  • SSH to the Lingoport Suite System
  • Install 'telnet', if it is not already present sudo yum install telnet
  • Validate that you can connect to your proxy server, while keeping proxy settings clear - for now:
http_proxy="" https_proxy="" telnet <proxyurl-no-http/s> <proxyport>
Example:
http_proxy="" https_proxy="" telnet masnes.testproxy.internal.lingoport.io 3128
Result:
Trying 172.31.81.146...
Connected to masnes.testproxy.internal.lingoport.io.
Escape character is '^]'.
Note:After connection is established, Enter will quit and return you to your terminal. If connection takes a while, use Ctrl-C instead.
If the connection takes more than 5 seconds, you may not be able to connect to the proxy. Please check your security groups / firewall / networking settings before continuing!
  • Next, as a comparison point, try curling to an external site that will return your ip address, without using the proxy settings yet.
Here, we use 'https://canhazip.com' as the external site. We set a timeout in case the connection is blocked.
Command
https_proxy="" curl https://canhazip.com --silent --connect-timeout 5 || echo "NOTE: ...Connection timed out after 5 seconds."
Expected Result
<your systems ip>, something like '34.195.141.162'
Alternate Expected Result - You'll see this if you are blocked from external connections by your firewall (possibly what you're using the proxy to avoid!):
NOTE ...Connection timed out after 5 seconds.
  • Next, try using the proxy with curl + an external site that will return your ip address.
Command
https_proxy="http://<proxyurl>:<proxyport>" curl https://canhazip.com --silent --connect-timeout 5 || echo "FAILED: With proxy, connection timed out after 5 seconds."
Example
https_proxy="http://masnes.testproxy.internal.lingoport.io:3128" curl https://canhazip.com --silent --connect-timeout 5 || echo "FAILED: With proxy, connection timed out after 5 seconds."
Expected Result
<proxy ip>, something different from the above. For the above example it was '54.205.127.98', which was different from the previous '34.195.141.162'. This time, the connection must not fail.

If you're able to connect to the proxy via telnet, you're able to use the proxy to get your remote ip from https://canhazip.com, and if the result you get from https://canhazip.com via the proxy is different than without the proxy, then you are ready to proceed.

Defining the Proxy for use with Localyzer Amazon Translate

To set a proxy server, define the environment variable "https_proxy" prior to running Localyzer with Amazon Translate. Through Jenkins, the easiest way to do so is:

Manage Jenkins -> (Goes to a new page)
Configure System -> (Goes to a new page)
Global Properties -> Environment Variables -> Add (Button at the bottom of the Environment Variables settings)

Settings should be: Name: https_proxy Value: http://<yourproxy>:<yourproxyport>

Example https_proxy http://masnes.testproxy.internal.lingoport.io:3128

Jenkins-config-env-https proxy.png

Click either 'Save' or 'Apply' once you are finished.

Validating the Proxy through Jenkins

You can add a step to your Jenkins Automation build(s) to confirm that the proxy is used. To do so:

  • Start at the Jenkins HomePage
  • Select the build you would like to test with. For example 'MyCompany.MyProject' -> (Goes to a new page)
  • Go to Configure on the left -> (Goes to a new page)
  • Scroll to the bottom of the Configure page. You should see an 'Add build step' button just above 'Post-build actions'
  • Add build step -> Execute shell -> (An empty 'Execute shell' box should appear)

Jenkins-add-build-step-execute-shell.png

  • In the box, add the following content:
set +e  # curl may fail without the proxy, depending on network configuration.
set +x  # optional setting. Don't show commands run, so output is clearer.
echo
echo "Validating Proxy is Used:"
echo
echo "Attempting connection without proxy. May timeout if network blocks traffic without proxy."
echo "Otherwise, should return the external ip of the system."
echo -n "IP returned: "
https_proxy="" curl https://canhazip.com --silent --connect-timeout 5 || echo "NOTE: Without proxy, connection timed out after 5 seconds."
echo
echo "Attempting connection with proxy. Proxy is valid if connection succeeds."
echo "Should return the external ip of the proxy server."
echo "    details: https_proxy set to: $https_proxy"
echo -n "IP returned: "
curl "https://canhazip.com" --silent --connect-timeout 5 || echo "FAILED: With the proxy set, connection timed out after 5 seconds."
echo
echo
  • After you have filled it out, drag and drop this box so that it is moved higher, above any other boxes such as 'Update Dashboard' or 'L10n Vendor'. This way it runs first.

Jenkins-proxy-build-step-execute-shell-above-run-lrm.png

  • Save the build. Then run it. Navigate to the console output of the build. You should see a section that shows:
Validating Proxy is Used:

Attempting connection without proxy. May timeout if network blocks traffic without proxy.
Otherwise, should return the external ip of the system.
IP returned: 34.195.141.162

Attempting connection with proxy. Proxy is valid if connection succeeds.
Should return the external ip of the proxy server.
    details: https_proxy set to: http://masnes.testproxy.internal.lingoport.io:3128
IP returned: 54.205.127.98

Like so:

Jenkins-proxy-build-output-validate-proxy.png


You are now using an https proxy!

Please see Machine Translation for details of Localyzer Machine Translation. Proxy configurations have been validated with Amazon Translate. It will take effect automatically as long as the https_proxy environment variable is set.