Difference between revisions of "Pull Requests"

From Lingoport Wiki
Jump to: navigation, search
m (GitLabl Pull Request)
(Pull Requests via Jenkins Running on Docker)
(37 intermediate revisions by 2 users not shown)
Line 1: Line 1:
  +
= Introduction =
Pull Requests, Merge Requests code changes can be analyzed using the Globalyzer Scanner.
 
  +
For GitHub, GitLab and Bitbucket, commits, Pull Requests / Merge Requests code changes can be analyzed using the Globalyzer Scanner.
   
  +
A Webhook lets a Jenkins job know to scan the code committed. Only the code modified is analyzed by the Globalyzer scanner. The result is a comment in GitHub, GitLab or Bitbucket showing issues detected.
= GitHub Pull Request =
 
See:
 
* https://vimeo.com/817611647
 
   
  +
In the example below, one line was modified and the Globalyzer scan shows an i18n issue. The developer can decide how to handle it.
= GitLab Pull Request =
 
See:
 
* https://vimeo.com/817610534
 
   
  +
[[File:PullRequestExample.png|600px|center]]
= To Be Edited =
 
   
  +
This is unobtrusive and provides actionable i18n information to developers.
   
* [[ GitHub Pull Requests | GitHub Pull Requests ]]
 
* [[ Distributed Jenkins | Distributed Jenkins ]]
 
The Lingoport Suite offers two options for Pull Requests from your repository. The Pull Request can be done on your Lingoport Server or there is a distributed option that uses the Master/Agent capability of Jenkins to distribute the work among several nodes.
 
   
  +
= Video Introduction to GitHub, GitLab, and Bitbucket =
To use the Distributed Pull Request option, please install the [[Lingoport_Suite_Installation#NodeInstaller|Node Installer]] optional software.
 
  +
Below are short introductions of the functionality around Globalyzer scanning for Git repository:
  +
  +
* '''GitHub''' https://vimeo.com/817611647
   
  +
* '''GitLab''' https://vimeo.com/817610534
= GitHub and Git Pull Requests=
 
GitHub is a Web-based Git repository hosting service. It offers all of the distributed revision control and source code management (SCM) functionality of Git as well as adding its own features.
 
   
  +
* '''Bitbucket''' https://vimeo.com/826230267
Files can be committed in a Git branch or directly in the Master branch. When committing files in a branch, pull requests let you know what changes you've made to a repository before they are committed to the repository's master branch. Once a pull request is sent, interested parties can review the set of changes, discuss potential modifications, and even push follow-up commits if necessary.
 
   
  +
= Pull Requests via Jenkins Running on Docker =
   
  +
== Prerequisites ==
[[File:PullRequest.png|800px]]
 
  +
<ol>
= Configuring Pull Requests =
 
  +
<li> VM running Linux and Docker
== Verify the Lingoport Dashboard Plugin ==
 
  +
<li> JQ JSON Parser - Statically compiled binary version
The Pull Requests rely on the Dashboard GitHub plugin. If your system was installed using the Stack Installer, this plugin should have been included. Verify that it is installed.
 
<ol>
+
<ol>
  +
<li> https://stedolan.github.io/jq/
<li> Go to the Lingoport Dashboard and log in as Administrator.
 
  +
</ol>
<li> Select the '''Administration''' tab at the top of the window.
 
  +
<li> GitHub PAT credentials to connect to GitHub
<li> Select the '''System''' pulldown and '''Update Center'''
 
  +
<ol>
<li> With '''Installed''' selected, search for '''GitHub'''. You should find a plugin with the description: "Provide some integration between GitHub and SonarQube"
 
  +
<li> The user for these PAT credentials has read access to the repository
</ol>
 
  +
</ol>
If you do not find the plugin installed, select '''Available''', search for 'GitHub' again and '''Install''' it.
 
  +
<li> Globalyzer.com credentials for logging in to globalyzer.com
  +
<ol>
  +
<li> username
  +
<li> password
  +
</ol>
  +
<li> Obtain a copy of Globalyzer Lite from https://www.globalyzer.com/gzserver/home/installclient (requires Globalyzer.com credentials)
  +
<li> A copy of your Globalyzer.license from globalyzer.com or your local globalyzer server instance.
  +
<li> Jenkins will use the following URL to download all plugins required during the installation. This URL is required to be reachable from the VM Jenkins is being installed on prior to starting the install.
  +
<ol>
  +
<li> https://updates.jenkins.io/update-center.json
  +
</li>
  +
</ol>
  +
<li> Optional:
  +
<ol>
  +
<li> Apache or nginx could be installed and running on the VM itself as a reverse proxy to front the Jenkins instance </li>
  +
</ol>
  +
</ol>
   
== Set up Nodes on Jenkins ==
+
== File System Configuration ==
  +
<p>
  +
If Apache is installed as per normal for many Lingoport systems, the following is required to configure Apache to permit proper connection to the Jenkins Docker container
  +
</p>
  +
<ol>
  +
<li> Create the directory structure for the volume storage used by jenkins. Be sure there is sufficient space. Base install will consume 650MB of space for the volume itself. </li>
  +
<code> mkdir ~/docker/jenkins/lingoport/bin </code>
  +
<li> Place the Globalyzer License into the lingoport directory created </li>
  +
<code> cp ~/PRC-Docker/Globalyzer.license ~/docker/jenkins/lingoport/ </code>
  +
<li> Copy the downloaded copy of Globalyzer Lite into the lingoport directory </li>
  +
<code> cp ~/PRC-Docker/globalyzer-lite-VERSION.zip ~/docker/jenkins/lingoport/ </code>
  +
<li> Copy the jq binary into the bin directory and set the permissions </li>
  +
<code>
  +
cp -p ~/PRC-Docker/jq ~/docker/jenkins/lingoport/bin/
  +
chmod +x ~/docker/jenkins/lingoport/bin/jq
  +
</code>
  +
<li> Unzip the Globalyzer Lite zip file in the lingoport directory and remove the zip file once complete </li>
  +
<code>
  +
cd ~/docker/jenkins/lingoport
  +
unzip globalyzer-lite-VERSION.zip
  +
rm ./globalyzer-lite-VERSION.zip
  +
</code>
  +
<li> Create the .globalyzerrc file in the jenkins directory, add the proper credentials and set the permissions </li>
  +
<code>
  +
vi .globalyzerrc
  +
chmod 600 .globalyzerrc
  +
</code>
  +
<li> Confirm the contents of the .globalyzerrc file are entered like this: (Note no equals or colon, just whitespace between key and value) </li>
  +
<code>
  +
username <GIT LOGIN USERNAME>
  +
password <GIT LOGIN PASSWORD>
  +
server <URL of Globalyzer Server>
  +
</code>
  +
</ol>
   
  +
== Launching the Jenkins Docker Container ==
If you are using the Distributed Pull Requests option, the first thing that needs to be done is to set up nodes in Jenkins. If you installed the [[Installing_the_Node_Installer|Node Installer]] and followed its setup instructions, there should be a master system (where Jenkins and Lingoport Dashboard are hosted) and at least one agent or node system. The master system should be able to ssh into the agents' centos user and the jenkins user. If this is not completed, make sure that is done before proceeding further.
 
   
  +
<p> Retrieve these two items as they are now required to proceed </p>
If you are not using nodes and agents, skip to the PullRequests section.
 
  +
<ol>
  +
<li> Obtain the UID and GID for the user that Docker is being ran by.
  +
<li> Identify the the port that Jenkins should be listening on.
  +
<ol>
  +
<li> In the example below it was determined to be 8085 </li>
  +
</ol>
  +
</li>
  +
</ol>
  +
<p> The next command line is an example of what would be used to start the docker instance for Jenkins. Here is an explanation of the switches being used when no proxy is required for the container to reach required destinations. </p>
  +
<p>
  +
The command string will create a new container instance that will automatically restart the container unless specifically
  +
stopped. It will also reuse the existing container on a restart of the OS. If the container is stopped before the
  +
reboot is performed using <i> sudo docker stop "CONTAINTER ID" </i>, it will need to be started using <i>sudo docker start "CONTAINER ID"</i>
  +
</p>
  +
<ol>
  +
<li> The -d detaches the command being run from the terminal. Normally this is something that will only be used
  +
once everything is in place and confirmed to be working. For initial startup, its use can be ignored. </li>
  +
<li> The -u is for setting the uid/gid of the user that owns the volume storage identified earlier. This user
  +
should also be the user that is starting the docker instance. </li>
  +
<li> The -p 8085:8080 is the port external to internal mapping identified earlier. The 8080 would never change,
  +
only the external port is modifiable </li>
  +
<li> The -p 50000:50000 is for mapping between Jenkins instances only. Not required for standalone systems. </li>
  +
<li> The JENKINS_OPTS sets the baseurl for the application </li>
  +
<li> The -v mounts the prior locally created file system to the running docker instance as a volume and mas it. </li>
  +
<li> The jenkins/jenkins:lts-jdk11 is the long term support image for Jenkins. Currently, it is Jenkins 2.387.3 </li>
  +
</ol>
  +
  +
<code>
  +
sudo docker run -d -u 1000:1000 -p 8085:8080 -p 50000:50000 \
  +
--env JENKINS_OPTS="--prefix=/jenkins" --restart=unless-stopped \
  +
-v /USER_HOME_DIR/docker/jenkins:/var/jenkins_home jenkins/jenkins:lts-jdk11
  +
</code>
   
  +
<p>
===Create Nodes in Jenkins===
 
  +
If a proxy is required to be leveraged by the container to reach destinations the next command line would be used to start the docker instance for Jenkins instead of the previous one. Note the additional env variables that are added to support the proxy.
On the Master system, in Jenkins, select <b>Manage Jenkins → Manage Nodes → New Node</b>
 
  +
</p>
   
  +
<code>
Set up each agent node using this example:
 
  +
sudo docker run -d -u 1000:1000 -p 8085:8080 -p 50000:50000 \
  +
--env JENKINS_OPTS="--prefix=/jenkins" \
  +
--env http_proxy="http://<PROXY_URL>:<PORT>" \
  +
--env https_proxy="http://<PROXY_URL>:<PORT>" \
  +
--restart=unless-stopped \
  +
-v /home/centos/docker/jenkins:/var/jenkins_home jenkins/jenkins:lts-jdk11
  +
</code>
   
  +
<p>
[[File:nodes_config.png|800px]]
 
  +
This will download the required images and start the container. There will be a message about initial password,
  +
make note of it as it will be used to log into Jenkins
  +
</p>
   
Select '''Add''' to create the jenkins ssh credentials
 
   
  +
[[File:Credentials.png|600px]]
 
  +
<code>
  +
*************************************************************
  +
*************************************************************
  +
*************************************************************
  +
Jenkins initial setup is required. An admin user has been created and a password generated.
  +
Please use the following password to proceed to installation:
  +
  +
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
  +
  +
This may also be found at: /var/jenkins_home/secrets/initialAdminPassword
  +
  +
*************************************************************
  +
*************************************************************
  +
*************************************************************
  +
</code>
   
  +
== Browser Connected Configuration ==
Nothing needs to be done with the master node. Online the new nodes and the console output should look like:
 
  +
<p> Go to application using the browser and use the above password to log in initially </p>
  +
[[File:Unlock-jenkins.png|noframe|500px]]
   
  +
<p> At this screen select Install Suggested Plugins </p>
<pre>
 
  +
[[File:Customize-jenkins.png|noframe|500px]]
[01/26/18 16:26:19] [SSH] Opening SSH connection to ec2-54-227-210-213.compute-1.amazonaws.com:22.
 
[01/26/18 16:26:19] [SSH] SSH host key matches key in Known Hosts file. Connection will be allowed.
 
[01/26/18 16:26:19] [SSH] Authentication successful.
 
[01/26/18 16:26:19] [SSH] The remote user's environment is:
 
BASH=/usr/bin/bash
 
BASHOPTS=cmdhist:extquote:force_fignore:hostcomplete:interactive_comments:progcomp:promptvars:sourcepath
 
....
 
</pre>
 
   
  +
<p> The plugins will begin to install, this will only be a few minutes. </p>
===Verify the Agent setup in Jenkins ===
 
  +
[[File:Getting-started-jenkins.png|noframe|500px]]
To verify that the Agent setup is correct, use the Jenkins 'Debug' job.
 
   
  +
<p> Create the Admin account. This will be a new account outside of what was used to connect initially. It will be the administrator account. </p>
Under '''General''' check '''Restrict where this project can be run''' and set it to '''lingoport'''.
 
  +
[[File:Create-first-admin-user-jenkins.png|noframe|500px]]
   
  +
<p> This sets up the URL with a path to the instance. It will include /jenkins so it should be something like “http(s)://<YOUR FQDN/jenkins/ </p>
In the '''Execute Shell''' leave only:
 
  +
[[File:Instance-config-jenkins.png|noframe|500px]]
<pre>
 
whoami
 
pwd
 
sleep 5
 
</pre>
 
   
  +
<p> The base Jenkins system install is complete at this point. </p>
Run the job. The console output should show:
 
  +
[[File:Jenkins-ready.png|noframe|500px]]
<pre>
 
Started by user anonymous
 
Building remotely on Agent1 (lingoport) in workspace /var/lib/jenkins/jenkins_slave/workspace/Debug
 
[Debug] $ /bin/sh -xe /tmp/jenkins7953464360000549139.sh
 
+ whoami
 
jenkins
 
+ pwd
 
/var/lib/jenkins/jenkins_slave/workspace/Debug
 
+ sleep 5
 
</pre>
 
   
  +
<p> Click Start using Jenkins and the Dashboard appears </p>
Note that it shows that it ran on the Agent using the lingoport label.
 
  +
[[File:Dashboard-jenkins.png|noframe|500px]]
   
  +
<p> The alert is caused by this running locally and not using nodes. Click on it and select “Dismiss” </p>
== Jenkins PullRequest and DistributedPullRequest jobs==
 
  +
[[File:Node-details-jenkins.png|noframe|500px]]
There are two Jenkins job templates for Distributed Pull Requests.
 
* Lingoport.SampleLite-PullRequest
 
* Lingoport.SampleLite-DistributedPullRequest
 
   
  +
<p> If once this is cleared, this message appears. It is due to a reverse proxy being used that needs to have some additional configuration. </p>
The Distributed Pull Request makes use of the Jenkins master/agent system.
 
  +
[[File:Proxy-error-jenkins.png|noframe|500px]]
   
  +
<p> When using Apache as a reverse proxy, adding these two entries to the virtual host SSL configuration should correct the error. </p>
Create a new Jenkins job using the '''Lingoport.SampleLite-PullRequest''' or '''Lingoport.SampleLite-DistributedPullRequest''' template. This should be modeled on an existing Jenkins project.
 
   
  +
<code>
For example, if you have a Jenkins project called '''Acme.BigProject''' and you want to use PullRequests with this project and repository, name the new project '''Acme.BigProject-PullRequest'''.
 
  +
RequestHeader set X-Forwarded-Proto "https"
  +
  +
RequestHeader set X-Forwarded-Port "443"
  +
</code>
   
  +
== Add GUI Configuration Items for Pull Requests and Commits ==
=== Configuring the PullRequest Jenkins job ===
 
   
  +
<p> Install and additional plugin called “Discard Old Builds” by going to Manage Jenkins → Manage Plugins → Available Plugins and search for “Discard old builds” </p>
==== Additional Configuration ====
 
  +
[[File:Plugins1-jenkins.png|noframe|500px]]
* /var/lib/jenkins/Lingoport_Data/Dashboard/github.properties
 
This file tells what username/token in GitHub will be used to push the data over. It should have three key/value pairs:
 
   
  +
<p> Install an additional plugin called “Throttle Concurrent Builds” by going to Manage Jenkins → Manage Plugins → Available Plugins and search for “Throttle Concurrent Builds”</p>
sonar.github.login=<myGitLogin>
 
  +
[[File:Plugins2-jenkins.png|noframe|500px]]
sonar.github.oauth=<created token>
 
sonar.host.url=<http(s)://<dashboard url>/
 
   
  +
<p> Once plugin is installed to go Manage Jenkins → Configure System → Throttle Concurrent Builds and select Add Category: </p>
* General: GitHub project / project Url
 
In the Source Code Management section of the Jenkins job configuration, the GitHub 'project url' text field needs a GitHub URL in the form
 
 
https://github.com/<organization>/<project>
 
   
  +
<p> Create Category Name PRC_THROTTLE and set Maximum Total Concurrent Builds = 1 and Maximum Concurrent Builds Per Node = 1. </p>
==== The Jenkins Job ====
 
Once you have created the new PullRequest project, some configurations need to be modified.
 
<ol>
 
<li> Make sure the name is '''<Group>.<Project>-PullRequest'''. The <Group>.<Project> Jenkins job should already exist and have been executed. Anything other than <Group>.<Project>-PullRequest may cause problems.
 
<li> Leave the '''payload''' parameter alone. Do not modify it.
 
<li> If nodes have been set up for other projects, select '''Restrict where this project can be run''' and set it to '''master'''. If nodes have NOT been set up, nothing needs to be done with this option.
 
<li> Under '''Build Triggers''', select '''Trigger builds remotely (e.g., from scripts)''' and set the value to '''HOOK'''.
 
<li> Under Build, and Execute Shell, set the Project Definition File for <Group>.<Project>
 
<pre>
 
LITE_PROJECT_DEFINITION=/var/lib/jenkins/jobs/<Group>.<Project>/workspace/lingoport/LiteProjectDefinition.xml
 
</pre>
 
<li> '''Save''' the configuration and '''Enable''' the project.
 
</ol>
 
From the command line console, in /var/lib/jenkins/Lingoport_Data/Dashboard/Projects/GROUPNAME.PROJECTNAME-PullRequest/ (this may need to be created), make a '''sonar-project-pr.properties''' file that looks like:
 
<div style="margin-left: 2em;">
 
<pre>
 
# Make sure a Dashboard user is set up with the credential below
 
# otherwise the dashboard will not accept the pushed data
 
#
 
# -------Using $DASHBOARD_HOME/sonar-project-template.properties
 
#
 
sonar.login=admin
 
sonar.password=<admin password>
 
#
 
# Do not change the properties below
 
# the capital letter tokens will be replaced by
 
# build scripts
 
#
 
sonar.projectKey=PullRequest:scan
 
sonar.projectName=PullRequest
 
sonar.projectVersion=PR
 
   
  +
<p> Select “Apply” to save </p>
sonar.language=lport
 
sonar.encoding=UTF-8
 
   
  +
<p> Set up the environment variables in Jenkins: </p>
sonar.sources=/var/lib/jenkins/jobs/<Group>.<Project>-PullRequest/workspace
 
sonar.lingoport.project.root=/var/lib/jenkins/Lingoport_Data/Dashboard/Projects/<Group>.<Project>-PullRequest
 
   
  +
<ol>
#
 
  +
<li> Select the Environment Variables checkbox
# For the GitHub SonarQube Plugin to push the results to GitHub PR
 
  +
<li> Select Add and add the following
# The same for all:
 
  +
<ol>
#
 
  +
<li> For a GitHub Installation
# -------Using $DASHBOARD_HOME/sonar-project-pr-template.properties
 
  +
<ol>
#
 
  +
<li> GITHUB_PRC_FILES = $JENKINS_HOME/lingoport/<globalyzer-lite-dir>/GitHub_PRC </li>
 
  +
</ol>
sonar.analysis.mode=preview
 
  +
<li> PATH = $PATH:$JENKINS_HOME/lingoport/bin
sonar.github.endpoint=https://api.github.com
 
  +
<li> If https is to be used for connecting to the git repos set to true. If ssh is to be used to connect to git repos set to false.
sonar.github.disableInlineComments=true
 
  +
<ol>
 
  +
<li> useHTTPS = true or false </li>
# Specific for this install
 
  +
</ol>
 
  +
</li>
# Specific to this GitHub Project
 
  +
<li> Optional - If Proxy is required:
# If the github.properties file does not exist, the job will not push
 
  +
<ol>
# the data to GitHub
 
  +
<li> Add entry _JAVA_OPTIONS with proxy details as shown in screenshot below </li>
#
 
  +
<li> Dhttp.proxyHost=<FQDN> Dhttp.proxyPort=<PORT> Dhttps.proxyHost=<FQDN> Dhttps.proxyPort=<PORT> </li>
# -------Using $LINGOPORT_DATA/Dashboard/github.properties
 
  +
</ol>
#
 
  +
</li>
sonar.github.login=<myGitLogin>
 
  +
</ol>
sonar.github.oauth=<created token>
 
  +
</li>
sonar.host.url=<http(s)://<dashboard url>/
 
  +
</ol>
 
  +
<br>
# Generated from the Jenkins setting under
 
  +
<p></p>
# ------ General: GitHub project / project Url
 
  +
[[File:Env-var1-jenkins.png|noframe|500px]]
sonar.github.repository=<Repo> #for example: NASA/nasa-latex-blocks
 
  +
<br>
# Generated from the Webhook payload
 
  +
[[File:Env-var2-jenkins.png|noframe|500px]]
sonar.github.pullRequest=<pr number>
 
</pre>
+
<br>
  +
[[File:Jenkins_proxy_for_java.png|noframe|500px]]
</div>
 
  +
<br>
  +
<p> Click Apply and Save </p>
  +
<br>
  +
<p> Add new API token to be used by GitHub Webhook </p>
  +
<p> In Jenkins create a new API token for the user that will be running the build. This will be required to be added to the webhook to permit connection back to the Jenkins instance </p>
  +
The credentials used for GitHub will require read access to the git repository.
  +
<ol>
  +
<li> Select Dashboard → Manage Jenkins → Manage Users → Select the gear icon on the right side of the Admin user (Admin user created earlier)
  +
<li> In the API Token section, select Add new Token
  +
<li> Set the name of the new token to GitHub-PRC-TOKEN or something that is memorable and can identify with the PRC action.
  +
<ol>
  +
<li> Be sure to copy the token value as it will never be displayed beyond this time. </li>
  +
</ol>
  +
</li>
  +
</ol>
  +
<p></p>
  +
[[File:Api-token-2.png|noframe|500px]]
   
  +
==Add File System Configuration Items for Pull Requests and Commits==
=== Configuring the DistributedPullRequest Jenkins job ===
 
  +
<p>Leave the GUI and go back to the file system </p>
Once you have created the new DistributedPullRequest project, some configurations need to be modified. The DistributedPullRequest should be based on a <Group>.<Project> Jenkins job that has been executed.
 
 
<ol>
 
<ol>
<li> Leave the '''payload''' parameter alone
+
<li> Edit the github.properties file:
  +
<ol>
<li> Set the '''GIT_REPO''' parameter. This should be set to the form '''Company/Project''' as seen on GitHub. For example, in GitHub, NASA has many projects so the GIT_REPO would be set to NASA/nasa-latex-blocks
 
  +
<li> /HOME_DIR/docker/jenkins/lingoport/globalyzer-lite-VERSION/GitHub_PRC/github.properties.
<li> Set the '''LITE_PROJECT_DEFINITION''' parameter to the value on the Agent. <code>$JENKINS_HOME/jenkins_slave/workspace/<Group>.<Project>-DistributedPullRequest/lingoport/LiteProjectDefinition.xml </code>
 
  +
<li> Update the github.login and github.oauth with the appropriate PAT values that were created ahead of time.
<li> Select '''Restrict where this project can be run''' and set it to '''lingoport'''.
 
  +
</li>
<li> Under '''Build Triggers''', select '''Trigger builds remotely (e.g., from scripts)''' and set the value to '''HOOK'''.
 
  +
</ol>
<li> '''Save''' the configuration and '''Enable''' the project.
 
  +
<li> When using https there will need to be a .netrc file placed in the jenkins_home directory inside of the container. This can be placed in the volume directory at /HOME_DIR/docker/jenkins/.netrc
  +
<ol>
  +
<li> login <GitHub login ID> (Not email)
  +
<li> password PAT
  +
</li>
  +
</ol>
  +
<li> When using ssh there will need to be the ssh public and private keys stored in the /HOME_DIR/docker/jenkins/.ssh directory. Also with ssh be sure to manually connect to the git repo prior to using the PRC scripts. The known_hosts file needs to be updated and that is a manual process that requires manually accepting the key. This should only be needed to be performed once.
  +
<li> Run the installer for Globalyzer-Lite. This must be done inside of the contains so everything is properly set up in that environment
  +
<ol>
  +
<li> Obtain the container id
  +
<ol>
  +
<li> <code> sudo docker container ps </code>
  +
</li>
  +
</ol>
  +
<li> <code> sudo docker exec -it CONTAINER ID /bin/bash </code>
  +
<li> <code> cd ~/lingoport/globalyzer-lite-VERSION </code>
  +
<li> <code>./install-lite.sh </code> (Should return Globalyzer Lite installation completed successfully)
  +
</li>
  +
</ol>
  +
</li>
 
</ol>
 
</ol>
   
  +
== Create Jenkins Build for Pull Requests and Commits via GitHub ==
== Configuring GitHub for PullRequests ==
 
   
Some configuration needs to be done on GitHub for the PullRequests to occur.
 
 
In the GitHub page for your Project:
 
 
<ol>
 
<ol>
<li>Select '''Settings''' at the top of the window.
+
<li> Log into Jenkins and go to the Dashboard
<li>Select the '''Webhooks''' tab in the left menu.
+
<li> Select New Item
  +
<li> Create an item name for the PRC build.
<li>Select the '''Add webhook''' button.
 
  +
<li> Use an item name that is memorable, something like GitHub-PRC is fine.
  +
<li> Select Freestyle Project
  +
<li> Select OK
  +
<li> Add a description (optional)
  +
<li> Select “This project is parameterized”
  +
<li> Add two string parameters
  +
<ol>
  +
<li> payload = NOT_SET
  +
<li> LITE_PROJECT_DEFINITION = DEFAULT
  +
<ol>
  +
<li> DEFAULT looks for the project definition to exist in the lingoport directory of the working branch as such: lingoport/LiteProjectDefinition.xml
  +
</li>
  +
</ol>
  +
<li> Select “Throttle Concurrent Builds”
  +
<ol>
  +
<li> Set Maximum Total Concurrent Builds to 1
  +
<li> Set Maximum Concurrent Builds Per Node to 1
  +
</li>
  +
</ol>
  +
<li> In Build Triggers Section
  +
<ol>
  +
<li> Select Trigger builds remotely (e.g., from scripts)
  +
<li> In the Authentication Token place a token you wish to use. Something like HOOK for example.
  +
</li>
  +
</ol>
  +
<li> In Build Steps Section
  +
<ol>
  +
<li> Select Execute Shell
  +
<li> Place the following shell code
  +
<ul>
  +
<li> <code> set +x </code>
  +
<li> <code> $GITHUB_PRC_FILES/job_lite_pr.sh $GITHUB_PRC_FILES/pr_message.html </code>
  +
</li>
  +
</ul>
  +
<li> In the Post-build Actions
  +
<ol>
  +
<li> Select Discard Old Builds
  +
<li> Set Days to keep builds to 14
  +
</li>
  +
</ol>
  +
<li> Save the configuration
  +
</li>
  +
</ol>
  +
</li>
 
</ol>
 
</ol>
   
  +
=== GitHub Configuration ===
   
  +
<p>
[[File:webhook.png|800px]]
 
  +
Webhooks are calls made from the GitHub side down to the Jenkins instance. They provide information required by the Jenkins build to properly scan the files that were either committed or had a pull request performed on them. This is discussed in detail in the Videos found in earlier sections of this page.
  +
</p>
  +
<p>
  +
Below is a summary of the steps.
  +
</p>
 
<ol>
 
<ol>
  +
<li> Log into GitHub
<li> Set the '''Payload URL''' to
 
  +
<li> Navigate to the repository that will have pull requests and commits applied to it
<pre>http://<Jenkins IP>/buildByToken/buildWithParameters?job=<Group>.<Project>-PullRequest&token=HOOK</pre>
 
  +
<li> Click on Settings → Webhooks→ Add webhook
for PullRequest jobs or
 
  +
<li> Enter the following:
<pre>http://<Jenkins IP>/buildByToken/buildWithParameters?job=<Group>.<Project>-DistributedPullRequest&token=HOOK</pre> for DistributedPullRequest jobs.
 
  +
<ol>
<li> Make sure the '''Content Type''' is <code>application/x-www-form-urlencoded</code>
 
  +
<li> Payload URL
<li> Leave '''Secret''' blank
 
  +
<ol>
<li> '''Which events would you like to trigger this webhook?''' Select: '''Let me select individual events''' and check the '''Pull Request''' item.
 
  +
<li> URL found in Build Triggers Section of the Jenkins Build
<li> Select '''Active'''.
 
  +
<li> It will likely require the API Token generated prior along with the username that the build in Jenkins was created
<li> '''Update''' or '''Add Webhook'''
 
  +
<li> ht<span>tps://USER:TOKEN@JENKINS_URL/jenkins/job/BUILD_NAME/buildWithParameters?token=BUILD_TOKEN</span>
  +
</li>
  +
</ol>
  +
<li> Content Type application/x-www-form-urlencoded
  +
<li> Radio button: Let me select individual events
  +
<ol>
  +
<li> Pushes
  +
<li> Pull Requests
  +
<li> Pushes, Pull Requests
  +
</li>
  +
</ol>
  +
</li>
  +
</ol>
  +
<li> Using the Recent Deliveries tab it's possible to check the status of a web hook call to confirm that it worked correctly.
  +
<ol>
  +
<li> There should be a build triggered for each webhook called submitted
  +
<li> If there are issues use the Redeliver button to resend
  +
</li>
  +
</ol>
  +
</li>
 
</ol>
 
</ol>
   
  +
=== Install Validation ===
If everything is correct, then after adding the Webhook, the Jenkins job <Group>.<Project>-PullRequest or <Group>.<Project>-DistributedPullRequest should be triggered. The job may FAIL, but at this point the verifying the communication between GitHub and the Jenkins job is a success.
 
  +
<p> This completes the basic installation. To confirm that everything is working correctly. </p>
 
= Verify the PullRequest =
 
To test that the Pull Request works,
 
 
<ol>
 
<ol>
  +
<li> Log into GitHub and navigate to a repository
<li> Use or create a new branch in the Git Project with changes to internationalization issues.
 
  +
<li> Make a change directly to a file in the GitHub repository and “Commit” the change (assuming Commits are enabled).
<li> In GitHub, select the '''Pull Requests''' tab
 
  +
<li> Switch to Jenkins and confirm that the build was triggered and successfully completed
<li> Select the '''New Pull Request''' button
 
  +
<li> Switch back to GitHub and click on the commit id for the modification in and confirm that there is an associated summary.
<li> Select the two branches you wish to compare. You should see the differences on the screen between the two branches.
 
  +
<li> Clicking on the File/Line link in the summary should show the code line and modification made.
<li> Select the '''Create New Pull Request''' button and enter a comment.
 
  +
</li>
<li> Select '''Create Pull Request'''
 
 
</ol>
 
</ol>
   
  +
<br>
Return to the Jenkins Pull Request job. It should be triggered and running. If it is successful, return to your Pull Request tab in GitHub and you should see something like:
 
  +
  +
<p> This is an image of the Analysis Summary in GitHub for a commit </p>
  +
[[File:Intall-validation-1.png|noframe|500px]]
  +
  +
<br>
  +
  +
<p> This image shows the line of code that is presented when clicking on the link in the Analysis Summary </p>
  +
[[File:Install-validation-2.png|noframe|500px]]
  +
  +
== SAML Integration (Optional) ==
  +
<p>
  +
To optionally leverage SAML in place of the local user login system, follow the instructions found on this page
  +
<p>
  +
  +
[[Docker Jenkins SAML Configuration]]
  +
  +
= Jenkins Upgrade when using Docker =
  +
  +
== Upgrade Required Indication ==
  +
  +
When logging into Jenkins, there will be a message on the Manage Jenkins page indicating that a new version is available.
  +
  +
[[file:Before_Upgrade.png |800px]]
  +
  +
== Upgrade Procedure ==
  +
  +
1. Log into the VM containing docker with a user that has sudo privileges.
  +
  +
2. Stop the existing jenkins/jenkins:lts-jdk11 container
  +
sudo docker stop <CONTAINER ID>
  +
  +
3. Make a backup of the existing Jenkins directory that is mounted as a volume. All Jenkins files are contained on the file system, there is no associated DB.
  +
$ cp -pr /home/<USER_DIR>/docker/jenkins /home/USER_DIR/docker/jenkins_<MMDDYYYY>
  +
  +
4. Pull the latest version of the jenkins/jenkins:lts-jdk11 from docker.io. Note that it will indicate a newer image has been downloaded.
  +
sudo docker pull jenkins/jenkins:lts-jdk11
  +
lts-jdk11: Pulling from jenkins/jenkins
  +
bd73737482dd: Pull complete
  +
747b9186aa97: Pull complete
  +
77e4b3fa9f98: Pull complete
  +
4185da018337: Pull complete
  +
fbfe8a536a91: Pull complete
  +
c28d8031aee7: Pull complete
  +
d90093f5fc84: Pull complete
  +
b7bf9221df23: Pull complete
  +
22a3392c99b4: Pull complete
  +
bbe8cad20a08: Pull complete
  +
c81331c127c6: Pull complete
  +
c125b4d0161b: Pull complete
  +
da080ad7c1d0: Pull complete
  +
Digest: sha256:90f7f78a7e114516216b6c0a06e00c597d6490597caec9d7bd7a95ce5c7dada0
  +
Status: Downloaded newer image for jenkins/jenkins:lts-jdk11
  +
docker.io/jenkins/jenkins:lts-jdk11
  +
  +
5. Start the new container with the updated image using one of the two options below depending upon whether a proxy is required to be leveraged from inside of the container or not.
  +
  +
'''Option #1''' - Run the newly updated container to start Jenkins with the latest
  +
version (Command is the same as the one used initially to run)
  +
  +
$ sudo docker run -d -u 1000:1000 -p 8085:8080 -p 50000:50000 \
  +
--env JENKINS_OPTS="--prefix=/jenkins" \
  +
--restart=unless-stopped \
  +
-v /home/centos/docker/jenkins:/var/jenkins_home jenkins/jenkins:lts-jdk11
  +
$
  +
$ sudo docker container ls
  +
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
  +
1924e0a6bf0d jenkins/jenkins:lts-jdk11 "/usr/bin/tini -- /u…" 2 minutes ago Up 2 minutes 0.0.0.0:50000->50000/tcp, :::50000->50000/tcp, 0.0.0.0:8085->8080/tcp, :::8085->8080/tcp peaceful_snyder
  +
  +
'''Option #2''' - Run the newly updated container to start Jenkins with
  +
the latest version of Jenkins when a proxy is needed for outgoing communications
  +
version (Command is the same as the one used initially to run)
  +
  +
$ sudo docker run -d -u 1000:1000 -p 8085:8080 -p 50000:50000 \
  +
--env JENKINS_OPTS="--prefix=/jenkins" \
  +
--env http_proxy="http://<PROXY_URL>:<PORT>" \
  +
--env https_proxy="http://<PROXY_URL>:<PORT>" \
  +
--restart=unless-stopped \
  +
-v /home/centos/docker/jenkins:/var/jenkins_home jenkins/jenkins:lts-jdk11
  +
$
  +
$ sudo docker container ls
  +
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
  +
9e2650b5ec1c jenkins/jenkins:lts-jdk11 "/usr/bin/tini -- /u…" 33 seconds ago Up 32 seconds 0.0.0.0:50000->50000/tcp, :::50000->50000/tcp, 0.0.0.0:8085->8080/tcp, :::8085->8080/tcp elegant_engelbart
  +
  +
  +
6. Confirm that the new version is running properly by logging in the web interface and going to the manage jenkins section. There should not be mention of a new version being available.
  +
  +
[[file:After_Upgrade.png |800px]]
  +
  +
7. At the bottom of this page, the current version will be displayed in the footer.
  +
  +
8. The old container can be removed if desired at this point.
  +
sudo docker rm <CONTAINER ID>
  +
  +
9. The old image can also be removed if desired at this point.
  +
sudo docker rmi <IMAGE ID>
  +
  +
10. The system will now be running the latest version of the Jenkins docker image and both the old images and containers will have been removed.
  +
  +
= Historical Version of the Pull Request/Commit Globalyzer Scanning =
   
  +
[[ Historical Pull Request | Historical Pull Request ]]
[[File:Pulltest.png|800px]]
 

Revision as of 13:23, 1 May 2024

Introduction

For GitHub, GitLab and Bitbucket, commits, Pull Requests / Merge Requests code changes can be analyzed using the Globalyzer Scanner.

A Webhook lets a Jenkins job know to scan the code committed. Only the code modified is analyzed by the Globalyzer scanner. The result is a comment in GitHub, GitLab or Bitbucket showing issues detected.

In the example below, one line was modified and the Globalyzer scan shows an i18n issue. The developer can decide how to handle it.

PullRequestExample.png

This is unobtrusive and provides actionable i18n information to developers.


Video Introduction to GitHub, GitLab, and Bitbucket

Below are short introductions of the functionality around Globalyzer scanning for Git repository:

Pull Requests via Jenkins Running on Docker

Prerequisites

  1. VM running Linux and Docker
  2. JQ JSON Parser - Statically compiled binary version
    1. https://stedolan.github.io/jq/
  3. GitHub PAT credentials to connect to GitHub
    1. The user for these PAT credentials has read access to the repository
  4. Globalyzer.com credentials for logging in to globalyzer.com
    1. username
    2. password
  5. Obtain a copy of Globalyzer Lite from https://www.globalyzer.com/gzserver/home/installclient (requires Globalyzer.com credentials)
  6. A copy of your Globalyzer.license from globalyzer.com or your local globalyzer server instance.
  7. Jenkins will use the following URL to download all plugins required during the installation. This URL is required to be reachable from the VM Jenkins is being installed on prior to starting the install.
    1. https://updates.jenkins.io/update-center.json
  8. Optional:
    1. Apache or nginx could be installed and running on the VM itself as a reverse proxy to front the Jenkins instance

File System Configuration

If Apache is installed as per normal for many Lingoport systems, the following is required to configure Apache to permit proper connection to the Jenkins Docker container

  1. Create the directory structure for the volume storage used by jenkins. Be sure there is sufficient space. Base install will consume 650MB of space for the volume itself.
  2.       mkdir ~/docker/jenkins/lingoport/bin 
    
  3. Place the Globalyzer License into the lingoport directory created
  4.       cp ~/PRC-Docker/Globalyzer.license ~/docker/jenkins/lingoport/ 
    
  5. Copy the downloaded copy of Globalyzer Lite into the lingoport directory
  6.       cp ~/PRC-Docker/globalyzer-lite-VERSION.zip ~/docker/jenkins/lingoport/ 
    
  7. Copy the jq binary into the bin directory and set the permissions
  8.       
           cp -p  ~/PRC-Docker/jq ~/docker/jenkins/lingoport/bin/
           chmod +x ~/docker/jenkins/lingoport/bin/jq
         
    
  9. Unzip the Globalyzer Lite zip file in the lingoport directory and remove the zip file once complete
  10.      
           cd ~/docker/jenkins/lingoport
           unzip globalyzer-lite-VERSION.zip
           rm ./globalyzer-lite-VERSION.zip
         
    
  11. Create the .globalyzerrc file in the jenkins directory, add the proper credentials and set the permissions
  12.      
           vi .globalyzerrc
           chmod 600 .globalyzerrc
         
    
  13. Confirm the contents of the .globalyzerrc file are entered like this: (Note no equals or colon, just whitespace between key and value)
  14.      
           username <GIT LOGIN USERNAME>
           password <GIT LOGIN PASSWORD>
           server <URL of Globalyzer Server> 
         
    

Launching the Jenkins Docker Container

Retrieve these two items as they are now required to proceed

  1. Obtain the UID and GID for the user that Docker is being ran by.
  2. Identify the the port that Jenkins should be listening on.
    1. In the example below it was determined to be 8085

The next command line is an example of what would be used to start the docker instance for Jenkins. Here is an explanation of the switches being used when no proxy is required for the container to reach required destinations.

The command string will create a new container instance that will automatically restart the container unless specifically stopped. It will also reuse the existing container on a restart of the OS. If the container is stopped before the reboot is performed using sudo docker stop "CONTAINTER ID" , it will need to be started using sudo docker start "CONTAINER ID"

  1. The -d detaches the command being run from the terminal. Normally this is something that will only be used once everything is in place and confirmed to be working. For initial startup, its use can be ignored.
  2. The -u is for setting the uid/gid of the user that owns the volume storage identified earlier. This user should also be the user that is starting the docker instance.
  3. The -p 8085:8080 is the port external to internal mapping identified earlier. The 8080 would never change, only the external port is modifiable
  4. The -p 50000:50000 is for mapping between Jenkins instances only. Not required for standalone systems.
  5. The JENKINS_OPTS sets the baseurl for the application
  6. The -v mounts the prior locally created file system to the running docker instance as a volume and mas it.
  7. The jenkins/jenkins:lts-jdk11 is the long term support image for Jenkins. Currently, it is Jenkins 2.387.3
 
   sudo docker run -d -u 1000:1000 -p 8085:8080 -p 50000:50000 \
     --env JENKINS_OPTS="--prefix=/jenkins" --restart=unless-stopped \
     -v /USER_HOME_DIR/docker/jenkins:/var/jenkins_home jenkins/jenkins:lts-jdk11
 

If a proxy is required to be leveraged by the container to reach destinations the next command line would be used to start the docker instance for Jenkins instead of the previous one. Note the additional env variables that are added to support the proxy.

 
   sudo docker run -d -u 1000:1000 -p 8085:8080 -p 50000:50000 \
     --env JENKINS_OPTS="--prefix=/jenkins" \
     --env http_proxy="http://<PROXY_URL>:<PORT>" \
     --env https_proxy="http://<PROXY_URL>:<PORT>" \
     --restart=unless-stopped \
     -v /home/centos/docker/jenkins:/var/jenkins_home jenkins/jenkins:lts-jdk11
 

This will download the required images and start the container. There will be a message about initial password, make note of it as it will be used to log into Jenkins


 
   *************************************************************
   *************************************************************
   *************************************************************
   Jenkins initial setup is required. An admin user has been created and a password generated.
   Please use the following password to proceed to installation:
                                                                
   xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
                                                                  
   This may also be found at: /var/jenkins_home/secrets/initialAdminPassword
                                                                
   *************************************************************
   *************************************************************
   *************************************************************
 

Browser Connected Configuration

Go to application using the browser and use the above password to log in initially

noframe

At this screen select Install Suggested Plugins

noframe

The plugins will begin to install, this will only be a few minutes.

noframe

Create the Admin account. This will be a new account outside of what was used to connect initially. It will be the administrator account.

noframe

This sets up the URL with a path to the instance. It will include /jenkins so it should be something like “http(s)://<YOUR FQDN/jenkins/

noframe

The base Jenkins system install is complete at this point.

noframe

Click Start using Jenkins and the Dashboard appears

noframe

The alert is caused by this running locally and not using nodes. Click on it and select “Dismiss”

noframe

If once this is cleared, this message appears. It is due to a reverse proxy being used that needs to have some additional configuration.

noframe

When using Apache as a reverse proxy, adding these two entries to the virtual host SSL configuration should correct the error.

RequestHeader set X-Forwarded-Proto "https"

RequestHeader set X-Forwarded-Port "443"

Add GUI Configuration Items for Pull Requests and Commits

Install and additional plugin called “Discard Old Builds” by going to Manage Jenkins → Manage Plugins → Available Plugins and search for “Discard old builds”

noframe

Install an additional plugin called “Throttle Concurrent Builds” by going to Manage Jenkins → Manage Plugins → Available Plugins and search for “Throttle Concurrent Builds”

noframe

Once plugin is installed to go Manage Jenkins → Configure System → Throttle Concurrent Builds and select Add Category:

Create Category Name PRC_THROTTLE and set Maximum Total Concurrent Builds = 1 and Maximum Concurrent Builds Per Node = 1.

Select “Apply” to save

Set up the environment variables in Jenkins:

  1. Select the Environment Variables checkbox
  2. Select Add and add the following
    1. For a GitHub Installation
      1. GITHUB_PRC_FILES = $JENKINS_HOME/lingoport/<globalyzer-lite-dir>/GitHub_PRC
    2. PATH = $PATH:$JENKINS_HOME/lingoport/bin
    3. If https is to be used for connecting to the git repos set to true. If ssh is to be used to connect to git repos set to false.
      1. useHTTPS = true or false
    4. Optional - If Proxy is required:
      1. Add entry _JAVA_OPTIONS with proxy details as shown in screenshot below
      2. Dhttp.proxyHost=<FQDN> Dhttp.proxyPort=<PORT> Dhttps.proxyHost=<FQDN> Dhttps.proxyPort=<PORT>


noframe
noframe
noframe

Click Apply and Save


Add new API token to be used by GitHub Webhook

In Jenkins create a new API token for the user that will be running the build. This will be required to be added to the webhook to permit connection back to the Jenkins instance

The credentials used for GitHub will require read access to the git repository.

  1. Select Dashboard → Manage Jenkins → Manage Users → Select the gear icon on the right side of the Admin user (Admin user created earlier)
  2. In the API Token section, select Add new Token
  3. Set the name of the new token to GitHub-PRC-TOKEN or something that is memorable and can identify with the PRC action.
    1. Be sure to copy the token value as it will never be displayed beyond this time.

noframe

Add File System Configuration Items for Pull Requests and Commits

Leave the GUI and go back to the file system

  1. Edit the github.properties file:
    1. /HOME_DIR/docker/jenkins/lingoport/globalyzer-lite-VERSION/GitHub_PRC/github.properties.
    2. Update the github.login and github.oauth with the appropriate PAT values that were created ahead of time.
  2. When using https there will need to be a .netrc file placed in the jenkins_home directory inside of the container. This can be placed in the volume directory at /HOME_DIR/docker/jenkins/.netrc
    1. login <GitHub login ID> (Not email)
    2. password PAT
  3. When using ssh there will need to be the ssh public and private keys stored in the /HOME_DIR/docker/jenkins/.ssh directory. Also with ssh be sure to manually connect to the git repo prior to using the PRC scripts. The known_hosts file needs to be updated and that is a manual process that requires manually accepting the key. This should only be needed to be performed once.
  4. Run the installer for Globalyzer-Lite. This must be done inside of the contains so everything is properly set up in that environment
    1. Obtain the container id
      1. sudo docker container ps
    2. sudo docker exec -it CONTAINER ID /bin/bash
    3. cd ~/lingoport/globalyzer-lite-VERSION
    4. ./install-lite.sh (Should return Globalyzer Lite installation completed successfully)

Create Jenkins Build for Pull Requests and Commits via GitHub

  1. Log into Jenkins and go to the Dashboard
  2. Select New Item
  3. Create an item name for the PRC build.
  4. Use an item name that is memorable, something like GitHub-PRC is fine.
  5. Select Freestyle Project
  6. Select OK
  7. Add a description (optional)
  8. Select “This project is parameterized”
  9. Add two string parameters
    1. payload = NOT_SET
    2. LITE_PROJECT_DEFINITION = DEFAULT
      1. DEFAULT looks for the project definition to exist in the lingoport directory of the working branch as such: lingoport/LiteProjectDefinition.xml
    3. Select “Throttle Concurrent Builds”
      1. Set Maximum Total Concurrent Builds to 1
      2. Set Maximum Concurrent Builds Per Node to 1
    4. In Build Triggers Section
      1. Select Trigger builds remotely (e.g., from scripts)
      2. In the Authentication Token place a token you wish to use. Something like HOOK for example.
    5. In Build Steps Section
      1. Select Execute Shell
      2. Place the following shell code
        • set +x
        • $GITHUB_PRC_FILES/job_lite_pr.sh $GITHUB_PRC_FILES/pr_message.html
      3. In the Post-build Actions
        1. Select Discard Old Builds
        2. Set Days to keep builds to 14
      4. Save the configuration

    GitHub Configuration

    Webhooks are calls made from the GitHub side down to the Jenkins instance. They provide information required by the Jenkins build to properly scan the files that were either committed or had a pull request performed on them. This is discussed in detail in the Videos found in earlier sections of this page.

    Below is a summary of the steps.

    1. Log into GitHub
    2. Navigate to the repository that will have pull requests and commits applied to it
    3. Click on Settings → Webhooks→ Add webhook
    4. Enter the following:
      1. Payload URL
        1. URL found in Build Triggers Section of the Jenkins Build
        2. It will likely require the API Token generated prior along with the username that the build in Jenkins was created
        3. https://USER:TOKEN@JENKINS_URL/jenkins/job/BUILD_NAME/buildWithParameters?token=BUILD_TOKEN
      2. Content Type application/x-www-form-urlencoded
      3. Radio button: Let me select individual events
        1. Pushes
        2. Pull Requests
        3. Pushes, Pull Requests
    5. Using the Recent Deliveries tab it's possible to check the status of a web hook call to confirm that it worked correctly.
      1. There should be a build triggered for each webhook called submitted
      2. If there are issues use the Redeliver button to resend

    Install Validation

    This completes the basic installation. To confirm that everything is working correctly.

    1. Log into GitHub and navigate to a repository
    2. Make a change directly to a file in the GitHub repository and “Commit” the change (assuming Commits are enabled).
    3. Switch to Jenkins and confirm that the build was triggered and successfully completed
    4. Switch back to GitHub and click on the commit id for the modification in and confirm that there is an associated summary.
    5. Clicking on the File/Line link in the summary should show the code line and modification made.


    This is an image of the Analysis Summary in GitHub for a commit

    noframe


    This image shows the line of code that is presented when clicking on the link in the Analysis Summary

    noframe

    SAML Integration (Optional)

    To optionally leverage SAML in place of the local user login system, follow the instructions found on this page

    Docker Jenkins SAML Configuration

    Jenkins Upgrade when using Docker

    Upgrade Required Indication

    When logging into Jenkins, there will be a message on the Manage Jenkins page indicating that a new version is available.

    Before Upgrade.png

    Upgrade Procedure

    1. Log into the VM containing docker with a user that has sudo privileges.

    2. Stop the existing jenkins/jenkins:lts-jdk11 container

      sudo docker stop <CONTAINER ID>
    

    3. Make a backup of the existing Jenkins directory that is mounted as a volume. All Jenkins files are contained on the file system, there is no associated DB.

      $ cp -pr /home/<USER_DIR>/docker/jenkins /home/USER_DIR/docker/jenkins_<MMDDYYYY>
     
    

    4. Pull the latest version of the jenkins/jenkins:lts-jdk11 from docker.io. Note that it will indicate a newer image has been downloaded.

           sudo docker pull jenkins/jenkins:lts-jdk11
           lts-jdk11: Pulling from jenkins/jenkins
           bd73737482dd: Pull complete
           747b9186aa97: Pull complete
           77e4b3fa9f98: Pull complete
           4185da018337: Pull complete
           fbfe8a536a91: Pull complete
           c28d8031aee7: Pull complete
           d90093f5fc84: Pull complete
           b7bf9221df23: Pull complete
           22a3392c99b4: Pull complete
           bbe8cad20a08: Pull complete
           c81331c127c6: Pull complete
           c125b4d0161b: Pull complete
           da080ad7c1d0: Pull complete
           Digest: sha256:90f7f78a7e114516216b6c0a06e00c597d6490597caec9d7bd7a95ce5c7dada0
           Status: Downloaded newer image for jenkins/jenkins:lts-jdk11
           docker.io/jenkins/jenkins:lts-jdk11
    

    5. Start the new container with the updated image using one of the two options below depending upon whether a proxy is required to be leveraged from inside of the container or not.

    Option #1 - Run the newly updated container to start Jenkins with the latest version (Command is the same as the one used initially to run)

       $ sudo docker run -d -u 1000:1000 -p 8085:8080 -p 50000:50000 \
         --env JENKINS_OPTS="--prefix=/jenkins" \
         --restart=unless-stopped \
         -v /home/centos/docker/jenkins:/var/jenkins_home jenkins/jenkins:lts-jdk11
       $
       $ sudo docker container ls
       CONTAINER ID   IMAGE                       COMMAND                  CREATED         STATUS         PORTS                                                                                      NAMES
       1924e0a6bf0d   jenkins/jenkins:lts-jdk11   "/usr/bin/tini -- /u…"   2 minutes ago   Up 2 minutes   0.0.0.0:50000->50000/tcp, :::50000->50000/tcp, 0.0.0.0:8085->8080/tcp, :::8085->8080/tcp   peaceful_snyder
    

    Option #2 - Run the newly updated container to start Jenkins with the latest version of Jenkins when a proxy is needed for outgoing communications version (Command is the same as the one used initially to run)

       $ sudo docker run -d -u 1000:1000 -p 8085:8080 -p 50000:50000 \
         --env JENKINS_OPTS="--prefix=/jenkins" \
         --env http_proxy="http://<PROXY_URL>:<PORT>" \
         --env https_proxy="http://<PROXY_URL>:<PORT>" \
         --restart=unless-stopped \
         -v /home/centos/docker/jenkins:/var/jenkins_home jenkins/jenkins:lts-jdk11
       $
       $ sudo docker container ls
       CONTAINER ID   IMAGE                       COMMAND                  CREATED          STATUS          PORTS                                                                                      NAMES
       9e2650b5ec1c   jenkins/jenkins:lts-jdk11   "/usr/bin/tini -- /u…"   33 seconds ago   Up 32 seconds   0.0.0.0:50000->50000/tcp, :::50000->50000/tcp, 0.0.0.0:8085->8080/tcp, :::8085->8080/tcp   elegant_engelbart
    


    6. Confirm that the new version is running properly by logging in the web interface and going to the manage jenkins section. There should not be mention of a new version being available.

    After Upgrade.png

    7. At the bottom of this page, the current version will be displayed in the footer.

    8. The old container can be removed if desired at this point.

           sudo docker rm <CONTAINER ID>
    

    9. The old image can also be removed if desired at this point.

           sudo docker rmi <IMAGE ID>
    

    10. The system will now be running the latest version of the Jenkins docker image and both the old images and containers will have been removed.

    Historical Version of the Pull Request/Commit Globalyzer Scanning

    Historical Pull Request