Olibouban: Created page with " '''THIS PAGE DESCRIBES PREVIOUS CONFIGURATION OF PULL REQUEST/COMMIT SCANNING WITH GLOBALYZER''' * GitHub Pull Requests * Distributed Jenkins..."

2023-05-26T21:09:44Z

Created page with " '''THIS PAGE DESCRIBES PREVIOUS CONFIGURATION OF PULL REQUEST/COMMIT SCANNING WITH GLOBALYZER''' * GitHub Pull Requests * Distributed Jenkins..."

New page

'''THIS PAGE DESCRIBES PREVIOUS CONFIGURATION OF PULL REQUEST/COMMIT SCANNING WITH GLOBALYZER'''

* [[ GitHub Pull Requests | GitHub Pull Requests ]]
* [[ Distributed Jenkins | Distributed Jenkins ]]
The Lingoport Suite offers two options for Pull Requests from your repository. The Pull Request can be done on your Lingoport Server or there is a distributed option that uses the Master/Agent capability of Jenkins to distribute the work among several nodes.

To use the Distributed Pull Request option, please install the [[Lingoport_Suite_Installation#NodeInstaller|Node Installer]] optional software.

= GitHub and Git Pull Requests=
GitHub is a Web-based Git repository hosting service. It offers all of the distributed revision control and source code management (SCM) functionality of Git as well as adding its own features.

Files can be committed in a Git branch or directly in the Master branch. When committing files in a branch, pull requests let you know what changes you've made to a repository before they are committed to the repository's master branch. Once a pull request is sent, interested parties can review the set of changes, discuss potential modifications, and even push follow-up commits if necessary.

[[File:PullRequest.png|800px]]
= Configuring Pull Requests =
== Verify the Lingoport Dashboard Plugin ==
The Pull Requests rely on the Dashboard GitHub plugin. If your system was installed using the Stack Installer, this plugin should have been included. Verify that it is installed.
<ol>
<li> Go to the Lingoport Dashboard and log in as Administrator.
<li> Select the '''Administration''' tab at the top of the window.
<li> Select the '''System''' pulldown and '''Update Center'''
<li> With '''Installed''' selected, search for '''GitHub'''. You should find a plugin with the description: "Provide some integration between GitHub and SonarQube"
</ol>
If you do not find the plugin installed, select '''Available''', search for 'GitHub' again and '''Install''' it.

== Set up Nodes on Jenkins ==

If you are using the Distributed Pull Requests option, the first thing that needs to be done is to set up nodes in Jenkins. If you installed the [[Installing_the_Node_Installer|Node Installer]] and followed its setup instructions, there should be a master system (where Jenkins and Lingoport Dashboard are hosted) and at least one agent or node system. The master system should be able to ssh into the agents' centos user and the jenkins user. If this is not completed, make sure that is done before proceeding further.

If you are not using nodes and agents, skip to the PullRequests section.

===Create Nodes in Jenkins===
On the Master system, in Jenkins, select <b>Manage Jenkins → Manage Nodes → New Node</b>

Set up each agent node using this example:

[[File:nodes_config.png|800px]]

Select '''Add''' to create the jenkins ssh credentials

[[File:Credentials.png|600px]]

Nothing needs to be done with the master node. Online the new nodes and the console output should look like:

<pre>
[01/26/18 16:26:19] [SSH] Opening SSH connection to ec2-54-227-210-213.compute-1.amazonaws.com:22.
[01/26/18 16:26:19] [SSH] SSH host key matches key in Known Hosts file. Connection will be allowed.
[01/26/18 16:26:19] [SSH] Authentication successful.
[01/26/18 16:26:19] [SSH] The remote user's environment is:
BASH=/usr/bin/bash
BASHOPTS=cmdhist:extquote:force_fignore:hostcomplete:interactive_comments:progcomp:promptvars:sourcepath
....
</pre>

===Verify the Agent setup in Jenkins ===
To verify that the Agent setup is correct, use the Jenkins 'Debug' job.

Under '''General''' check '''Restrict where this project can be run''' and set it to '''lingoport'''.

In the '''Execute Shell''' leave only:
<pre>
whoami
pwd
sleep 5
</pre>

Run the job. The console output should show:
<pre>
Started by user anonymous
Building remotely on Agent1 (lingoport) in workspace /var/lib/jenkins/jenkins_slave/workspace/Debug
[Debug] $ /bin/sh -xe /tmp/jenkins7953464360000549139.sh
+ whoami
jenkins
+ pwd
/var/lib/jenkins/jenkins_slave/workspace/Debug
+ sleep 5
</pre>

Note that it shows that it ran on the Agent using the lingoport label.

== Jenkins PullRequest and DistributedPullRequest jobs==
There are two Jenkins job templates for Distributed Pull Requests.
* Lingoport.SampleLite-PullRequest
* Lingoport.SampleLite-DistributedPullRequest

The Distributed Pull Request makes use of the Jenkins master/agent system.

Create a new Jenkins job using the '''Lingoport.SampleLite-PullRequest''' or '''Lingoport.SampleLite-DistributedPullRequest''' template. This should be modeled on an existing Jenkins project.

For example, if you have a Jenkins project called '''Acme.BigProject''' and you want to use PullRequests with this project and repository, name the new project '''Acme.BigProject-PullRequest'''.

=== Configuring the PullRequest Jenkins job ===

==== Additional Configuration ====
* /var/lib/jenkins/Lingoport_Data/Dashboard/github.properties
This file tells what username/token in GitHub will be used to push the data over. It should have three key/value pairs:

sonar.github.login=<myGitLogin>
sonar.github.oauth=<created token>
sonar.host.url=<http(s)://<dashboard url>/

* General: GitHub project / project Url
In the Source Code Management section of the Jenkins job configuration, the GitHub 'project url' text field needs a GitHub URL in the form

https://github.com/<organization>/<project>

==== The Jenkins Job ====
Once you have created the new PullRequest project, some configurations need to be modified.
<ol>
<li> Make sure the name is '''<Group>.<Project>-PullRequest'''. The <Group>.<Project> Jenkins job should already exist and have been executed. Anything other than <Group>.<Project>-PullRequest may cause problems.
<li> Leave the '''payload''' parameter alone. Do not modify it.
<li> If nodes have been set up for other projects, select '''Restrict where this project can be run''' and set it to '''master'''. If nodes have NOT been set up, nothing needs to be done with this option.
<li> Under '''Build Triggers''', select '''Trigger builds remotely (e.g., from scripts)''' and set the value to '''HOOK'''.
<li> Under Build, and Execute Shell, set the Project Definition File for <Group>.<Project>
<pre>
LITE_PROJECT_DEFINITION=/var/lib/jenkins/jobs/<Group>.<Project>/workspace/lingoport/LiteProjectDefinition.xml
</pre>
<li> '''Save''' the configuration and '''Enable''' the project.
</ol>
From the command line console, in /var/lib/jenkins/Lingoport_Data/Dashboard/Projects/GROUPNAME.PROJECTNAME-PullRequest/ (this may need to be created), make a '''sonar-project-pr.properties''' file that looks like:
<div style="margin-left: 2em;">
<pre>
# Make sure a Dashboard user is set up with the credential below
# otherwise the dashboard will not accept the pushed data
#
# -------Using $DASHBOARD_HOME/sonar-project-template.properties
#
sonar.login=admin
sonar.password=<admin password>
#
# Do not change the properties below
# the capital letter tokens will be replaced by
# build scripts
#
sonar.projectKey=PullRequest:scan
sonar.projectName=PullRequest
sonar.projectVersion=PR

sonar.language=lport
sonar.encoding=UTF-8

sonar.sources=/var/lib/jenkins/jobs/<Group>.<Project>-PullRequest/workspace
sonar.lingoport.project.root=/var/lib/jenkins/Lingoport_Data/Dashboard/Projects/<Group>.<Project>-PullRequest

#
# For the GitHub SonarQube Plugin to push the results to GitHub PR
# The same for all:
#
# -------Using $DASHBOARD_HOME/sonar-project-pr-template.properties
#

sonar.analysis.mode=preview
sonar.github.endpoint=https://api.github.com
sonar.github.disableInlineComments=true

# Specific for this install

# Specific to this GitHub Project
# If the github.properties file does not exist, the job will not push
# the data to GitHub
#
# -------Using $LINGOPORT_DATA/Dashboard/github.properties
#
sonar.github.login=<myGitLogin>
sonar.github.oauth=<created token>
sonar.host.url=<http(s)://<dashboard url>/

# Generated from the Jenkins setting under
# ------ General: GitHub project / project Url
sonar.github.repository=<Repo> #for example: NASA/nasa-latex-blocks
# Generated from the Webhook payload
sonar.github.pullRequest=<pr number>
</pre>
</div>

=== Configuring the DistributedPullRequest Jenkins job ===
Once you have created the new DistributedPullRequest project, some configurations need to be modified. The DistributedPullRequest should be based on a <Group>.<Project> Jenkins job that has been executed.
<ol>
<li> Leave the '''payload''' parameter alone
<li> Set the '''GIT_REPO''' parameter. This should be set to the form '''Company/Project''' as seen on GitHub. For example, in GitHub, NASA has many projects so the GIT_REPO would be set to NASA/nasa-latex-blocks
<li> Set the '''LITE_PROJECT_DEFINITION''' parameter to the value on the Agent. <code>$JENKINS_HOME/jenkins_slave/workspace/<Group>.<Project>-DistributedPullRequest/lingoport/LiteProjectDefinition.xml </code>
<li> Select '''Restrict where this project can be run''' and set it to '''lingoport'''.
<li> Under '''Build Triggers''', select '''Trigger builds remotely (e.g., from scripts)''' and set the value to '''HOOK'''.
<li> '''Save''' the configuration and '''Enable''' the project.
</ol>

== Configuring GitHub for PullRequests ==

Some configuration needs to be done on GitHub for the PullRequests to occur.

In the GitHub page for your Project:
<ol>
<li>Select '''Settings''' at the top of the window.
<li>Select the '''Webhooks''' tab in the left menu.
<li>Select the '''Add webhook''' button.
</ol>

[[File:webhook.png|800px]]
<ol>
<li> Set the '''Payload URL''' to
<pre>http://<Jenkins IP>/buildByToken/buildWithParameters?job=<Group>.<Project>-PullRequest&token=HOOK</pre>
for PullRequest jobs or
<pre>http://<Jenkins IP>/buildByToken/buildWithParameters?job=<Group>.<Project>-DistributedPullRequest&token=HOOK</pre> for DistributedPullRequest jobs.
<li> Make sure the '''Content Type''' is <code>application/x-www-form-urlencoded</code>
<li> Leave '''Secret''' blank
<li> '''Which events would you like to trigger this webhook?''' Select: '''Let me select individual events''' and check the '''Pull Request''' item.
<li> Select '''Active'''.
<li> '''Update''' or '''Add Webhook'''
</ol>

If everything is correct, then after adding the Webhook, the Jenkins job <Group>.<Project>-PullRequest or <Group>.<Project>-DistributedPullRequest should be triggered. The job may FAIL, but at this point the verifying the communication between GitHub and the Jenkins job is a success.

= Verify the PullRequest =
To test that the Pull Request works,
<ol>
<li> Use or create a new branch in the Git Project with changes to internationalization issues.
<li> In GitHub, select the '''Pull Requests''' tab
<li> Select the '''New Pull Request''' button
<li> Select the two branches you wish to compare. You should see the differences on the screen between the two branches.
<li> Select the '''Create New Pull Request''' button and enter a comment.
<li> Select '''Create Pull Request'''
</ol>

Return to the Jenkins Pull Request job. It should be triggered and running. If it is successful, return to your Pull Request tab in GitHub and you should see something like:

[[File:Pulltest.png|800px]]

Historical Pull Request - Revision history

Olibouban: Created page with " '''THIS PAGE DESCRIBES PREVIOUS CONFIGURATION OF PULL REQUEST/COMMIT SCANNING WITH GLOBALYZER''' * GitHub Pull Requests * Distributed Jenkins..."