Difference between revisions of "Distributed Jenkins"

From Lingoport Wiki
Jump to: navigation, search
Line 25: Line 25:
 
The 'jenkins' user must be created with the same home directory as the master 'jenkins' user, i.e. <code/var/lib/jenkins</code>.
 
The 'jenkins' user must be created with the same home directory as the master 'jenkins' user, i.e. <code/var/lib/jenkins</code>.
   
  +
== Node Installer ==
 
Make sure the following are installed with the correct version of Java for the given release:
 
Make sure the following are installed with the correct version of Java for the given release:
 
* Java
 
* Java
Line 38: Line 39:
 
* A template <code>pullrequest-sonar-project.properties</code> under /var/lib/jenkins/lingoport/templates
 
* A template <code>pullrequest-sonar-project.properties</code> under /var/lib/jenkins/lingoport/templates
 
** A number of items need to be set in the templates itself, including GitHub's username/OAuth Token, Dashboard Username / Password for all the project to be handled.
 
** A number of items need to be set in the templates itself, including GitHub's username/OAuth Token, Dashboard Username / Password for all the project to be handled.
  +
  +
The directory structure should look like after install on the slave node:
  +
  +
[jenkins@ip-172-31-45-177 jenkins]$ pwd
  +
/var/lib/jenkins
  +
[jenkins@ip-172-31-45-177 jenkins]$ ls -l
  +
total 4
  +
drwxrwxr-x. 3 jenkins jenkins 93 Dec 22 23:53 installers
  +
drwxrwxr-x. 4 jenkins jenkins 61 Dec 29 10:02 jenkins_slave
  +
drwxrwxr-x. 6 jenkins jenkins 82 Dec 27 23:29 lingoport
  +
[
  +
  +
[jenkins@ip-172-31-45-177 lingoport]$ pwd
  +
/var/lib/jenkins/lingoport
  +
[jenkins@ip-172-31-45-177 lingoport]$ ls -l
  +
total 4
  +
drwxrwxr-x. 2 jenkins jenkins 35 Dec 28 00:07 bin
  +
drwxr-xr-x. 5 jenkins jenkins 4096 Dec 22 21:24 globalyzer-lite-5.2
  +
drwxr-xr-x. 5 jenkins jenkins 37 Sep 20 09:57 sonar-scanner-2.8
  +
drwxrwxr-x. 2 jenkins jenkins 49 Dec 28 00:19 templates
   
   

Revision as of 18:10, 29 December 2016

Notes on the Jenkins Deployment for Scalability around Pull Requests from GitHub.

Close to: http://devopscube.com/setup-slaves-on-jenkins-2/

  1. on master:

Generate an id_rsa key, for example:

 ssh-keygen -t rsa -C "The access key for Jenkins slaves" 

make sure the chmod's are right

Credentials for Jenkins based on SSH username with Private Key: jenkins dashboard –> credentials –> Global credentials –> add credentials

Choose from file with the default id_rsa / id_rsa.pub

copy the public key

Important: This is all based on the key under /var/lib/jenkins/.ssh/ being with the following owner and access permissions:

  -rw-------.  1 jenkins jenkins 1675 Dec 21 21:20 id_rsa
  -rw-r--r--.  1 jenkins jenkins  419 Dec 21 21:20 id_rsa.pub

(see credentials below based on file)

  1. on slave

Jenkins User

The 'jenkins' user must be created with the same home directory as the master 'jenkins' user, i.e. <code/var/lib/jenkins.

Node Installer

Make sure the following are installed with the correct version of Java for the given release:

  • Java
  • git
  • Globalyzer Lite under ~jenkins/lingoport/globalyzer-lite-x.y
  • .globalyzerrc
  • sonar-scanner under jenkins/lingoport/sonar-scanner-x.y/
    • the configuration file must point back to the Dashboard Server!
  • jq (see for example https://www.linkedin.com/pulse/how-install-jq-centos-7-artur-todeschini)
    • yum install epel-release
    • yum install jq
  • The remote scripts, distributed_lite_pr.sh ...)
  • A template pullrequest-sonar-project.properties under /var/lib/jenkins/lingoport/templates
    • A number of items need to be set in the templates itself, including GitHub's username/OAuth Token, Dashboard Username / Password for all the project to be handled.

The directory structure should look like after install on the slave node:

[jenkins@ip-172-31-45-177 jenkins]$ pwd /var/lib/jenkins [jenkins@ip-172-31-45-177 jenkins]$ ls -l total 4 drwxrwxr-x. 3 jenkins jenkins 93 Dec 22 23:53 installers drwxrwxr-x. 4 jenkins jenkins 61 Dec 29 10:02 jenkins_slave drwxrwxr-x. 6 jenkins jenkins 82 Dec 27 23:29 lingoport [

[jenkins@ip-172-31-45-177 lingoport]$ pwd /var/lib/jenkins/lingoport [jenkins@ip-172-31-45-177 lingoport]$ ls -l total 4 drwxrwxr-x. 2 jenkins jenkins 35 Dec 28 00:07 bin drwxr-xr-x. 5 jenkins jenkins 4096 Dec 22 21:24 globalyzer-lite-5.2 drwxr-xr-x. 5 jenkins jenkins 37 Sep 20 09:57 sonar-scanner-2.8 drwxrwxr-x. 2 jenkins jenkins 49 Dec 28 00:19 templates


copy the public key (id_rsa.pub) into

.ssh/authorized_keys

make sure the chmod is 600, as in:

-rw-------. 1 jenkins jenkins  831 Dec 21 23:05 authorized_keys

Note: Make sure in the copy you have one line for the copy. Sometimes, copying may break the key into multiple lines.

  1. In AWS console / the master and slave nodes:
 Open the 22 ports to and from the two systems to allow jobs to run from Master to Slaves/Agents/Nodes
 Make sure the Outgoing ports and the Incoming ports for the Master and Slaves are allowed for 9000/other to allow Dashboard upload.
  1. On Slave:
  1. On Master:

When click OK. The Node should go on-line. The top of the log should look like:

 [12/22/16 15:47:13] [SSH] Opening SSH connection to 54.89.231.113:22.
 [12/22/16 15:47:13] [SSH] Authentication successful.
 [12/22/16 15:47:13] [SSH] The remote users environment is:
 

On your master machine go to Manage Jenkins > Manage Nodes.