Difference between revisions of "GitHub Pull Requests"

From Lingoport Wiki
Jump to: navigation, search
(Pull Request Analysis Configuration)
(How to Configure Pull Request Analysis)
(47 intermediate revisions by 3 users not shown)
Line 5: Line 5:
 
Files can be committed in a Git branch or directly in Master. When committing files in a branch, pull requests let you know what changes you've made to a repository before they are committed to the repository's master branch. Once a pull request is sent, interested parties can review the set of changes, discuss potential modifications, and even push follow-up commits if necessary.
 
Files can be committed in a Git branch or directly in Master. When committing files in a branch, pull requests let you know what changes you've made to a repository before they are committed to the repository's master branch. Once a pull request is sent, interested parties can review the set of changes, discuss potential modifications, and even push follow-up commits if necessary.
   
You can configure Globalyzer to scan the files that changed between the two branches of the pull request, and post a Globalyzer Analysis Summary on the pull request in Github.
+
You can configure Globalyzer to scan the files that changed between the two branches of a pull request, and post a Globalyzer Analysis Summary on the pull request in Github.
   
 
== Example of a Pull Request Analyzed with Globalyzer ==
 
== Example of a Pull Request Analyzed with Globalyzer ==
   
[[File:PrSummaryAnalysis.png]]
+
[[File:PrNewSummaryAnalysis.png]]
   
This is pull request <b>#44</b>. The change to the code in the pull request was to add multiple embedded strings and a locale-sensitive method to the files <code>ExampleMain.java</code> & <code>ExamplePanel.java</code>. This pull request is reporting 25 different issues, however since the <code>sonar-project.properties</code> attribute is set to <code>sonar.github.disableInlineComments=true</code>, the top ten issues (sorted by severity, by component, then by line) are put into a summary analysis as a pull request comment.
+
This is pull request <b>#8</b>. The change to the code in the pull request was to add a file containing one embedded string, one locale-sensitive method, one general pattern, and one static file reference to illustrate the different sections of the analysis summary. At the top of the analysis, are issue totals. Then the issues are displayed by type, with the top (up to) ten issues displayed.
   
Globalyzer analysis flagged these lines as critical issues. Clicking on the link next to the issue number will send the user to the line of code in that respective file.
+
Clicking on the file#line link will send the user to the line of code in that respective file. If an issue has associated help, as in the case for most locale-sensitive methods, the issue will appear as a link and clicking the issue will display the associated help.
   
Before merging the code back to master, the developer would typically externalize these strings, fix the locale-sensitive method issue, and re-submit the changed code / resource files for further analysis.
+
Before merging the code back to master, the developer would typically externalize embedded strings, fix locale-sensitive method issues, general pattern issues, and static file references and re-submit the changed code for further analysis.
   
  +
==Pre-requisites==
  +
You will need read access to the github repository. If using ssh to clone, follow [https://docs.github.com/en/authentication/connecting-to-github-with-ssh/about-ssh github's guide] to set up ssh clone access for an account which may be used for the pull request analysis.
   
  +
To check and see if you have read access to the github repository, try to clone from the command line:
== Pull Request Analysis Configuration ==
 
  +
  +
<code>git clone git@github.com:<RepoName>.git </code>
  +
  +
If you see:
  +
<code>fatal: Could not read from remote repository.
  +
Please make sure you have the correct access rights and the repository exists.</code> follow the [https://docs.github.com/en/authentication/connecting-to-github-with-ssh/about-ssh github's guide] to correct the issue before moving forward.
  +
  +
Github will need to be able to send HTTPS (port 443) requests which reach your system. See github's guide to github IP addresses [https://docs.github.com/en/enterprise-cloud@latest/authentication/keeping-your-account-and-data-secure/about-githubs-ip-addresses here], and check with your IT and networking departments to set up access or forwarding of Github's connections.
  +
  +
Once configured, you may verify that the pull request notifications are making it to the target machine via:
  +
  +
<code>sudo tail -f /var/log/httpd/access_log</code>
  +
  +
An example might look like:
  +
  +
<code>140.82.115.102 - - [18/Nov/2021:17:58:40 +0000] "POST /jenkins/buildByToken/buildWithParameters/build?job=CET-WorldWindExplorer-PullRequest&token=HOOK HTTP/1.1" 201 - "-" "GitHub-Hookshot/93dfdaa"</code>
  +
  +
==How to Configure Pull Request Analysis==
 
<ol>
 
<ol>
<li><b>Configure github.properties</b><br>
+
<li><b>Configure github.properties</b></li>
  +
 
You must create the file <code>/var/lib/jenkins/Lingoport_Data/Dashboard/github.properties</code> with the following contents:
 
You must create the file <code>/var/lib/jenkins/Lingoport_Data/Dashboard/github.properties</code> with the following contents:
* github.login=<your github login>
+
github.login=<your github login>
* github.oauth=<your github token>
+
github.oauth=<your github token>
  +
<b>Note:</b> To create a token for yourself on Github, navigate to https://github.com, select '''Settings''' from login dropdown, select '''Developer settings''' on left, select '''Personal access tokens''', click the '''Generate new token''' button. Also note that your access token must have the repo scope selected.
To create a token for yourself on Github:
 
* browse to https://github.com
 
* select <b>Settings</b> from login dropdown
 
* select <b>Developer Settings</b> on left
 
* select <b>Personal access tokens</b>
 
* click <b>Generate new token</b>
 
* fill out token information
 
* click <b>Generate token</b>
 
</li>
 
<li><b>Create a Pull Request Jenkins Job</b><br>
 
Create a new Jenkins job named <b><YourRepo>-PullRequest</b> by copying the existing job RepoName-PullRequest.
 
   
  +
[[File:PullRequestRepos.png|600px]]
You do not need to configure this job, just create it.
 
</li>
 
<li><b>Create Web Hook for Github Repository</b></li><
 
   
  +
<li><b>Create a Pull Request Jenkins Job</b></li>
The Pull Request on GitHub needs to trigger a Web hook back to Lingoport's Jenkins instance. To do so:
 
  +
  +
Create a new Jenkins job named <b><YourOrganization>.<YourRepoName>-PullRequest</b> by copying the template '''Lingoport.RepoName-PullRequest''' job.<br>
  +
Look at the configuration for your new job and verify that <b>Trigger builds remotely</b> is checked and then set the authentication token; the default is HOOK. The token is the only item you need to configure for your new job.
  +
  +
<b>Note:</b> Your Jenkins user must have special permissions to configure the PullRequest job. If your job does not have the <b>Trigger builds remotely</b> option, it is likely that your Jenkins user does not have the necessary permissions. Please see [[LRM_Jenkins_Configure_System#Configure_Global_Security|here]] for information on Jenkins Global Security Configuration.
  +
  +
<li><b>Create a Webhook for your Github Repository</b></li>
  +
  +
The Pull Request on GitHub needs to trigger a Webhook back to Lingoport's Jenkins instance. To do so:
 
* Navigate to the GitHub repository as an '''Admin''' account
 
* Navigate to the GitHub repository as an '''Admin''' account
* In Settings tab for the GitHub repository (available if the account has '''Admin''' permissions), add a Webhook
+
* In '''Settings''' tab for the GitHub repository (available if the account has '''Admin''' permissions), select '''Webhooks''' and the '''Add webhook''' button.
* The Web hook should follow this pattern:
+
* The Webhook should follow this pattern:
   
Payload URL: http(s)://JENKINS_URL/jenkins/buildByToken/buildWithParameters/build?job=RepoName-PullRequest&token=HOOK
+
Payload URL: http(s)://<b>JENKINS_URL</b>/jenkins/buildByToken/buildWithParameters/build?job=<b>Group.RepoName-PullRequest</b>&token=<b>HOOK</b>
   
 
Content type: application/x-www-form-urlencoded
 
Content type: application/x-www-form-urlencoded
Line 57: Line 75:
 
[[File:GitHubWebhook.png|500px]]
 
[[File:GitHubWebhook.png|500px]]
   
  +
* Also, select: <b>Let me select individual events.</b>
* Test the Web hook and especially check for firewall issues going back to the Jenkins system
 
  +
** Then: <b>Pull Requests</b>
  +
  +
* Test the Webhook and especially check for firewall issues going back to the Jenkins system. After creating the Webhook, it will attempt to send a payload which should trigger the Jenkins job. While still in '''Webhooks''', select the '''Recent Deliveries''' tab.
  +
** A response of '''201''' is successful and this can be verified by checking the Jenkins job. It should have been triggered and have some information in it.
  +
** A response of '''404''' means that there was a communications issue with Jenkins. Verify that port 443 is open. Once changes have been made, select the '''Redeliver''' button to test the payload again.
  +
  +
[[File:WebhookSuccessful.jpg|800px]]
  +
  +
   
 
</ol>
 
</ol>

Revision as of 23:56, 2 December 2021

GitHub

GitHub is a Web-based Git repository hosting service. It offers all of the distributed revision control and source code management (SCM) functionality of Git as well as adding its own features.

Git Pull Requests

Files can be committed in a Git branch or directly in Master. When committing files in a branch, pull requests let you know what changes you've made to a repository before they are committed to the repository's master branch. Once a pull request is sent, interested parties can review the set of changes, discuss potential modifications, and even push follow-up commits if necessary.

You can configure Globalyzer to scan the files that changed between the two branches of a pull request, and post a Globalyzer Analysis Summary on the pull request in Github.

Example of a Pull Request Analyzed with Globalyzer

PrNewSummaryAnalysis.png

This is pull request #8. The change to the code in the pull request was to add a file containing one embedded string, one locale-sensitive method, one general pattern, and one static file reference to illustrate the different sections of the analysis summary. At the top of the analysis, are issue totals. Then the issues are displayed by type, with the top (up to) ten issues displayed.

Clicking on the file#line link will send the user to the line of code in that respective file. If an issue has associated help, as in the case for most locale-sensitive methods, the issue will appear as a link and clicking the issue will display the associated help.

Before merging the code back to master, the developer would typically externalize embedded strings, fix locale-sensitive method issues, general pattern issues, and static file references and re-submit the changed code for further analysis.

Pre-requisites

You will need read access to the github repository. If using ssh to clone, follow github's guide to set up ssh clone access for an account which may be used for the pull request analysis.

To check and see if you have read access to the github repository, try to clone from the command line:

git clone git@github.com:<RepoName>.git

If you see: fatal: Could not read from remote repository. Please make sure you have the correct access rights and the repository exists. follow the github's guide to correct the issue before moving forward.

Github will need to be able to send HTTPS (port 443) requests which reach your system. See github's guide to github IP addresses here, and check with your IT and networking departments to set up access or forwarding of Github's connections.

Once configured, you may verify that the pull request notifications are making it to the target machine via:

sudo tail -f /var/log/httpd/access_log

An example might look like:

140.82.115.102 - - [18/Nov/2021:17:58:40 +0000] "POST /jenkins/buildByToken/buildWithParameters/build?job=CET-WorldWindExplorer-PullRequest&token=HOOK HTTP/1.1" 201 - "-" "GitHub-Hookshot/93dfdaa"

How to Configure Pull Request Analysis

  1. Configure github.properties
  2. You must create the file /var/lib/jenkins/Lingoport_Data/Dashboard/github.properties with the following contents:

      github.login=<your github login>
      github.oauth=<your github token>
    

    Note: To create a token for yourself on Github, navigate to https://github.com, select Settings from login dropdown, select Developer settings on left, select Personal access tokens, click the Generate new token button. Also note that your access token must have the repo scope selected.

    PullRequestRepos.png

  3. Create a Pull Request Jenkins Job
  4. Create a new Jenkins job named <YourOrganization>.<YourRepoName>-PullRequest by copying the template Lingoport.RepoName-PullRequest job.
    Look at the configuration for your new job and verify that Trigger builds remotely is checked and then set the authentication token; the default is HOOK. The token is the only item you need to configure for your new job.

    Note: Your Jenkins user must have special permissions to configure the PullRequest job. If your job does not have the Trigger builds remotely option, it is likely that your Jenkins user does not have the necessary permissions. Please see here for information on Jenkins Global Security Configuration.

  5. Create a Webhook for your Github Repository
  6. The Pull Request on GitHub needs to trigger a Webhook back to Lingoport's Jenkins instance. To do so:

    • Navigate to the GitHub repository as an Admin account
    • In Settings tab for the GitHub repository (available if the account has Admin permissions), select Webhooks and the Add webhook button.
    • The Webhook should follow this pattern:
      Payload URL: http(s)://JENKINS_URL/jenkins/buildByToken/buildWithParameters/build?job=Group.RepoName-PullRequest&token=HOOK
    
      Content type: application/x-www-form-urlencoded
    
      Pull Request event selected
    
      Active
    

    For example:

    GitHubWebhook.png

    • Also, select: Let me select individual events.
      • Then: Pull Requests
    • Test the Webhook and especially check for firewall issues going back to the Jenkins system. After creating the Webhook, it will attempt to send a payload which should trigger the Jenkins job. While still in Webhooks, select the Recent Deliveries tab.
      • A response of 201 is successful and this can be verified by checking the Jenkins job. It should have been triggered and have some information in it.
      • A response of 404 means that there was a communications issue with Jenkins. Verify that port 443 is open. Once changes have been made, select the Redeliver button to test the payload again.

    WebhookSuccessful.jpg