Node Installer - Revision history

Masnes at 00:00, 3 January 2018

2018-01-03T00:00:29Z

← Older revision		Revision as of 00:00, 3 January 2018
Line 16:		Line 16:

	=== On the Master system ===		=== On the Master system ===
	* Download the Lingoport-node-installer-x.y.z.zip, where x.y.z is the version number on a system which has access to the agent nodes and the master ~~node~~; Typically, download Lingoport-node-installer-x.y.z.zip on the Master ~~node~~.		* Download the Lingoport-node-installer-x.y.z.zip, where x.y.z is the version number on a system which has access to the agent nodes and the master system; Typically, download Lingoport-node-installer-x.y.z.zip on the Master system.
	* Unzip the zip file (on the Master ~~node~~ most likely)		* Unzip the zip file (on the Master system most likely)
	* If ansible is not installed, run install_ansible.sh		* If ansible is not installed, run install_ansible.sh
	* Go through the README.md file to configure the setup and run the ansible playbook. Make sure to have the right .ssh/authorized_keys even in Master as the [master] host needs to be filled in the hosts configuration file. So in effect, Master will ssh into Master; you can append the /home/centos/.ssh/rsa_id.pub into /home/centos/.ssh/authorized_keys to do so.		* Go through the README.md file to configure the setup and run the ansible playbook. Make sure to have the right .ssh/authorized_keys even in Master as the [master] host needs to be filled in the hosts configuration file. So in effect, Master will ssh into Master; you can append the /home/centos/.ssh/rsa_id.pub into /home/centos/.ssh/authorized_keys to do so.

Masnes: /* On the Master node */

2018-01-02T23:57:21Z

On the Master node

← Older revision		Revision as of 23:57, 2 January 2018
Line 15:		Line 15:
	-rw-------. 1 centos centos 814 Jan 12 22:28 authorized_keys		-rw-------. 1 centos centos 814 Jan 12 22:28 authorized_keys

	=== On the Master ~~node~~ ===		=== On the Master system ===
	* Download the Lingoport-node-installer-x.y.z.zip, where x.y.z is the version number on a system which has access to the agent nodes and the master node; Typically, download Lingoport-node-installer-x.y.z.zip on the Master node.		* Download the Lingoport-node-installer-x.y.z.zip, where x.y.z is the version number on a system which has access to the agent nodes and the master node; Typically, download Lingoport-node-installer-x.y.z.zip on the Master node.
	* Unzip the zip file (on the Master node most likely)		* Unzip the zip file (on the Master node most likely)

Masnes: /* Rationale */

2018-01-02T23:51:08Z

Rationale

← Older revision		Revision as of 23:51, 2 January 2018
Line 3:		Line 3:
	See https://wiki.jenkins-ci.org/display/JENKINS/Distributed+builds		See https://wiki.jenkins-ci.org/display/JENKINS/Distributed+builds

	To start with this model, Lingoport is putting together a Node Installer for agents which relies on a Master ~~node~~ installed with the Stack Installer; The first remote job is a Pull Request (PR) analysis for GitHub, so ~~that analyzing many~~ PR~~'s can~~ scale up.		To start with this model, Lingoport is putting together a Node Installer for agents which relies on a Master system installed with the Stack Installer; The first remote job is performs [[GitHub_Pull_Requests \| Pull Request (PR) analysis for GitHub]]. A large number of GitHub pull requests may be performed concurrently, so the ability to scale is important. Distributing the PR job between multiple systems is a means to scale it.

	== Agent Installation ==		== Agent Installation ==

Rdaly: /* On the Master nodes */

2017-12-20T18:25:02Z

On the Master nodes

← Older revision		Revision as of 18:25, 20 December 2017
Line 15:		Line 15:
	-rw-------. 1 centos centos 814 Jan 12 22:28 authorized_keys		-rw-------. 1 centos centos 814 Jan 12 22:28 authorized_keys

	=== On the Master ~~nodes~~ ===		=== On the Master node ===
	* Download the Lingoport-node-installer-x.y.z.zip, where x.y.z is the version number on a system which has access to the agent nodes and the master node; Typically, download Lingoport-node-installer-x.y.z.zip on the Master node.		* Download the Lingoport-node-installer-x.y.z.zip, where x.y.z is the version number on a system which has access to the agent nodes and the master node; Typically, download Lingoport-node-installer-x.y.z.zip on the Master node.
	* Unzip the zip file (on the Master node most likely)		* Unzip the zip file (on the Master node most likely)

Rdaly: /* Slave Node Installation */

2017-12-20T18:10:47Z

Slave Node Installation

Rdaly: /* Rationale */

2017-12-20T18:02:48Z

Rationale

← Older revision		Revision as of 18:02, 20 December 2017
Line 1:		Line 1:
	== Rationale ==		== Rationale ==
	Jenkins is setup by default on one system and executes all the jobs locally to that system. The Jenkins platform has the ability to distribute jobs to remote systems. This is called a ~~Master Slave~~ configuration in Jenkins parlance.		Jenkins is setup by default on one system and executes all the jobs locally to that system. The Jenkins platform has the ability to distribute jobs to remote systems. This is called a master/agent configuration in Jenkins parlance.
	See https://wiki.jenkins-ci.org/display/JENKINS/Distributed+builds		See https://wiki.jenkins-ci.org/display/JENKINS/Distributed+builds

	To start with this model, Lingoport is putting together a Node Installer for ~~slaves~~ which relies on a Master node installed with the Stack Installer; The first remote job is a Pull Request (PR) analysis for GitHub, so that analyzing many PR's can scale up.		To start with this model, Lingoport is putting together a Node Installer for agents which relies on a Master node installed with the Stack Installer; The first remote job is a Pull Request (PR) analysis for GitHub, so that analyzing many PR's can scale up.

	== Slave Node Installation ==		== Slave Node Installation ==

Olibouban at 00:28, 13 January 2017

2017-01-13T00:28:00Z

← Older revision		Revision as of 00:28, 13 January 2017
Line 55:		Line 55:
	** The job <b>must</b> be using the remote label "Restrict where this project can be run". The remote label, for instance <code>lingoport</code>, was set up when adding the new node.		** The job <b>must</b> be using the remote label "Restrict where this project can be run". The remote label, for instance <code>lingoport</code>, was set up when adding the new node.
	* In the Jenkins UI, the Build Execution Status widget should show "Master" and a number of slave nodes based on the nodes added so far.		* In the Jenkins UI, the Build Execution Status widget should show "Master" and a number of slave nodes based on the nodes added so far.

			== The Pull Request Distributed Jobs ==
			The first Lingoport job to be enabled is the GitHub Pull Request job. To configure it, copy the Lingoport.SampleLite-GitHubPullRequest job to your specific job, for instance Acme.FunWidgets-GitHubPullRequest. Make sure it is restricted to run on slave nodes identified by the label you created for those nodes, for instance <code>lingoport</code>.

Olibouban at 23:55, 12 January 2017

2017-01-12T23:55:35Z

← Older revision		Revision as of 23:55, 12 January 2017
Line 6:		Line 6:

	== Slave Node Installation ==		== Slave Node Installation ==
			=== On the machines for slave nodes ===
	* Setup the Linux CentOS systems to be the slave nodes.		* Setup the Linux CentOS systems to be the slave nodes.
	* In the sudo user of the slave nodes, add the id_rsa.pub from Master to ~/.ssh/authorized_keys (See ssh-keygen -t rsa). This will allow Master to ssh into the Slaves, which is a pre-requisites before the node installer can be run. The following is on a CentOS AWS sytem, where 'centos' is a user with 'sudo' privileges:		* In the sudo user of the slave nodes, add the id_rsa.pub from Master to ~/.ssh/authorized_keys (See ssh-keygen -t rsa). This will allow Master to ssh into the Slaves, which is a pre-requisites before the node installer can be run. The following is on a CentOS AWS sytem, where 'centos' is a user with 'sudo' privileges:
Line 14:		Line 15:
	-rw-------. 1 centos centos 814 Jan 12 22:28 authorized_keys		-rw-------. 1 centos centos 814 Jan 12 22:28 authorized_keys

			=== On the Master nodes ===
	* Download the Lingoport-node-installer-x.y.z.zip, where x.y.z is the version number on a system which has access to the slave nodes and the master node; Typically, download Lingoport-node-installer-x.y.z.zip on the Master node.		* Download the Lingoport-node-installer-x.y.z.zip, where x.y.z is the version number on a system which has access to the slave nodes and the master node; Typically, download Lingoport-node-installer-x.y.z.zip on the Master node.
	* Unzip the zip file (on the Master node most likely)		* Unzip the zip file (on the Master node most likely)
Line 30:		Line 32:
	PLAY [install pre-requisite programs to nodes]		PLAY [install pre-requisite programs to nodes]
	[...]		[...]

			== Subsequent Slave Node Installation from Master ==
			Once the installation has been run once, you can repeat the installation (from Master) by simply changing the hosts file to point to other slave nodes to add. They must have been configured as above.

	== Adding the new node(s) to Master ==		== Adding the new node(s) to Master ==

Olibouban: /* Adding the new node(s) to Master */

2017-01-12T23:52:05Z

Adding the new node(s) to Master

← Older revision		Revision as of 23:52, 12 January 2017
Line 49:		Line 49:
	* Check a job can be sent to the nodes: For instance, run a Debug job configured with some simple command like pwd and env.		* Check a job can be sent to the nodes: For instance, run a Debug job configured with some simple command like pwd and env.
	** The job <b>must</b> be using the remote label "Restrict where this project can be run". The remote label, for instance <code>lingoport</code>, was set up when adding the new node.		** The job <b>must</b> be using the remote label "Restrict where this project can be run". The remote label, for instance <code>lingoport</code>, was set up when adding the new node.
			* In the Jenkins UI, the Build Execution Status widget should show "Master" and a number of slave nodes based on the nodes added so far.

Olibouban: /* Adding the new node(s) to Master */

2017-01-12T23:50:10Z

Adding the new node(s) to Master

← Older revision		Revision as of 23:50, 12 January 2017
Line 47:		Line 47:
	Evacuated stdout		Evacuated stdout
	<b>Agent successfully connected and online</b>		<b>Agent successfully connected and online</b>
			* Check a job can be sent to the nodes: For instance, run a Debug job configured with some simple command like pwd and env.
			** The job <b>must</b> be using the remote label "Restrict where this project can be run". The remote label, for instance <code>lingoport</code>, was set up when adding the new node.

← Older revision		Revision as of 18:10, 20 December 2017
Line 5:		Line 5:
	To start with this model, Lingoport is putting together a Node Installer for agents which relies on a Master node installed with the Stack Installer; The first remote job is a Pull Request (PR) analysis for GitHub, so that analyzing many PR's can scale up.		To start with this model, Lingoport is putting together a Node Installer for agents which relies on a Master node installed with the Stack Installer; The first remote job is a Pull Request (PR) analysis for GitHub, so that analyzing many PR's can scale up.

	== ~~Slave Node~~ Installation ==		== Agent Installation ==
	=== On the machines for ~~slave~~ nodes ===		=== On the machines for agent nodes ===
	* Setup the Linux CentOS systems to be the ~~slave~~ nodes.		* Setup the Linux CentOS systems to be the agent nodes.
	* In the sudo user of the ~~slave~~ nodes, add the id_rsa.pub from Master to ~/.ssh/authorized_keys (See ssh-keygen -t rsa). This will allow Master to ssh into the ~~Slaves~~, which is a pre-requisites before the node installer can be run. The following is on a CentOS AWS sytem, where 'centos' is a user with 'sudo' privileges:		* In the sudo user of the agent nodes, add the id_rsa.pub from Master to ~/.ssh/authorized_keys (See ssh-keygen -t rsa). This will allow Master to ssh into the Agents, which is a pre-requisites before the node installer can be run. The following is on a CentOS AWS sytem, where 'centos' is a user with 'sudo' privileges:

	[centos@ip-172-31-37-148 .ssh]$ pwd		[centos@ip-172-31-37-148 .ssh]$ pwd
Line 16:		Line 16:

	=== On the Master nodes ===		=== On the Master nodes ===
	* Download the Lingoport-node-installer-x.y.z.zip, where x.y.z is the version number on a system which has access to the ~~slave~~ nodes and the master node; Typically, download Lingoport-node-installer-x.y.z.zip on the Master node.		* Download the Lingoport-node-installer-x.y.z.zip, where x.y.z is the version number on a system which has access to the agent nodes and the master node; Typically, download Lingoport-node-installer-x.y.z.zip on the Master node.
	* Unzip the zip file (on the Master node most likely)		* Unzip the zip file (on the Master node most likely)
	* If ansible is not installed, run install_ansible.sh		* If ansible is not installed, run install_ansible.sh
	* Go through the README.md file to configure the setup and run the ansible playbook. Make sure to have the right .ssh/authorized_keys even in Master as the [master] host needs to be filled in the hosts configuration file. So in effect, Master will ssh into Master; you can append the /home/centos/.ssh/rsa_id.pub into /home/centos/.ssh/authorized_keys to do so.		* Go through the README.md file to configure the setup and run the ansible playbook. Make sure to have the right .ssh/authorized_keys even in Master as the [master] host needs to be filled in the hosts configuration file. So in effect, Master will ssh into Master; you can append the /home/centos/.ssh/rsa_id.pub into /home/centos/.ssh/authorized_keys to do so.
	** The following checks for the communication between Master and ~~Slave~~(s)		** The following checks for the communication between Master and Agent(s)

	[centos@ip-172-31-60-142 node-installer]$ ansible-playbook -i hosts pre-allow-master-agent-ssh.yml		[centos@ip-172-31-60-142 node-installer]$ ansible-playbook -i hosts pre-allow-master-agent-ssh.yml
Line 27:		Line 27:
	[...]		[...]

	** The following sets up the ~~slave~~ node using the master's node configuration and Lingoport installed components:		** The following sets up the agent node using the master's node configuration and Lingoport installed components:

	[centos@ip-172-31-60-142 node-installer]$ ansible-playbook -i hosts setup-agents.yml		[centos@ip-172-31-60-142 node-installer]$ ansible-playbook -i hosts setup-agents.yml