Git to AWS S3 System - Revision history

Olibouban: Olibouban moved page AWS S3 VCS Configuration to Git to AWS S3 System

2023-12-12T15:32:13Z

Olibouban moved page AWS S3 VCS Configuration to Git to AWS S3 System

← Older revision	Revision as of 15:32, 12 December 2023
(No difference)

Olibouban: /* Jenkins/Dashboard Localyzer Project Configuration (Unsupported) */

2023-12-12T15:30:50Z

Jenkins/Dashboard Localyzer Project Configuration (Unsupported)

← Older revision		Revision as of 15:30, 12 December 2023
Line 220:		Line 220:
	=== Jenkins/Dashboard Localyzer Project Configuration (Unsupported) ===		=== Jenkins/Dashboard Localyzer Project Configuration (Unsupported) ===
	This section used to apply to Jenkins and Dashboard, i.e. versions up to Japan of our product line.		This section used to apply to Jenkins and Dashboard, i.e. versions up to Japan of our product line.
	This is not supported any longer.		'''This is not supported any longer.'''

	* [[On-Boarding_New_Projects\|On-Board your Project]], (but set the VCS details to 'None').		* [[On-Boarding_New_Projects\|On-Board your Project]], (but set the VCS details to 'None').

Olibouban: /* Command Center AWS S3 Data Source Configuration */

2023-12-12T15:30:33Z

Command Center AWS S3 Data Source Configuration

← Older revision		Revision as of 15:30, 12 December 2023
Line 218:		Line 218:
	* https://wiki.lingoport.com/index.php?title=AWS_S3_Data_Source_Credential		* https://wiki.lingoport.com/index.php?title=AWS_S3_Data_Source_Credential

	=== Jenkins/Dashboard Localyzer Project Configuration (~~Deprecated~~) ===		=== Jenkins/Dashboard Localyzer Project Configuration (Unsupported) ===
			This section used to apply to Jenkins and Dashboard, i.e. versions up to Japan of our product line.
			This is not supported any longer.

	* [[On-Boarding_New_Projects\|On-Board your Project]], (but set the VCS details to 'None').		* [[On-Boarding_New_Projects\|On-Board your Project]], (but set the VCS details to 'None').

Olibouban: /* Localyzer Project Configuration */

2023-12-12T15:29:29Z

Localyzer Project Configuration

← Older revision		Revision as of 15:29, 12 December 2023
Line 214:		Line 214:
	<code>bash -x</code> will echo each line before executing it; In particular, the variable assignment will be visible.		<code>bash -x</code> will echo each line before executing it; In particular, the variable assignment will be visible.

	= Localyzer Project Configuration =		= Command Center AWS S3 Data Source Configuration =
			See:
			* https://wiki.lingoport.com/index.php?title=AWS_S3_Data_Source_Credential

			=== Jenkins/Dashboard Localyzer Project Configuration (Deprecated) ===

	* [[On-Boarding_New_Projects\|On-Board your Project]], (but set the VCS details to 'None').		* [[On-Boarding_New_Projects\|On-Board your Project]], (but set the VCS details to 'None').

Rwilliams: Added section for capturing a log of the S3 wrapper script and automatically rotating the logs.

2022-12-19T14:04:21Z

Added section for capturing a log of the S3 wrapper script and automatically rotating the logs.

@@ Line 178: / Line 178: @@
      sudo systemctl start crond
      sudo systemctl status crond
 == Debugging ==

Olibouban at 16:43, 14 October 2022

2022-10-14T16:43:48Z

← Older revision		Revision as of 16:43, 14 October 2022
Line 5:		Line 5:

	To do so, the following will be needed:		To do so, the following will be needed:
	* '''Git/S3 System:''' one Linux system will host the ''bash'' scripts to automate the transfer from Git to S3 and back, most likely inside the customers network.		* '''Git/S3 System:''' one '''Linux''' system will host the ''bash'' scripts to automate the transfer from Git to S3 and back, most likely inside the customers network.
	* '''Lingoport System:''' The system hosting Jenkins, Dashboard, etc., connected to S3 and the TMS		* '''Lingoport System:''' The system hosting Jenkins, Dashboard, etc., connected to S3 and the TMS
	* '''A dedicated S3 bucket:''' The S3 bucket will have two main top directories:		* '''A dedicated S3 bucket:''' The S3 bucket will have two main top directories:

Olibouban: /* Localyzer Project Configuration */

2022-08-24T17:23:56Z

Localyzer Project Configuration

← Older revision		Revision as of 17:23, 24 August 2022
Line 195:		Line 195:

	* The configuration requires three parameters to be set. Make sure the <code><S3 URL></code>, <code>to_localyzer</code> and <code>from_localyzer</code> are the same as set up on the '''Git to S3 System'''.		* The configuration requires three parameters to be set. Make sure the <code><S3 URL></code>, <code>to_localyzer</code> and <code>from_localyzer</code> are the same as set up on the '''Git to S3 System'''.
	VCS_TYPE=S3		VCS_TYPE=s3
	S3_TO_LOCALYZER=s3://<S3 URL>/to_localyzer/<RepoName>_<branchName>		S3_TO_LOCALYZER=s3://<S3 URL>/to_localyzer/<RepoName>_<branchName>
	S3_FROM_LOCALYZER=s3://<S3 URL>/from_localyzer/<RepoName>_<branchName>		S3_FROM_LOCALYZER=s3://<S3 URL>/from_localyzer/<RepoName>_<branchName>

Olibouban: /* s3config.properties */

2022-08-24T16:58:52Z

s3config.properties

← Older revision		Revision as of 16:58, 24 August 2022
Line 130:		Line 130:
	S3_TO_LOCALYZER=s3://<S3 URL>/to_localyzer/		S3_TO_LOCALYZER=s3://<S3 URL>/to_localyzer/
	S3_FROM_LOCALYZER=s3://<S3 URL>/from_localyzer/		S3_FROM_LOCALYZER=s3://<S3 URL>/from_localyzer/
	* Set the DATA_DIR location for the working directories		* Set the DATA_DIR location for the working directories. The directory must exist and be readable/writable.
	DATA_DIR="/full/path/git_s3_data"		DATA_DIR="/full/path/git_s3_data"
	* Set the commit message for the translated files going back to the repository:		* Set the commit message for the translated files going back to the repository:
	COMMIT_MESSAGE="Localyzer Git-S3"		COMMIT_MESSAGE="Localyzer Git-S3"


	===repositories.txt===		===repositories.txt===

Olibouban: /* Configuration */

2022-08-24T16:56:14Z

Configuration

← Older revision		Revision as of 16:56, 24 August 2022
Line 125:		Line 125:
	Follow the README.md instructions.		Follow the README.md instructions.

	~~Mostly~~
	* ''s3config.properties''		=== s3config.properties===
	** Set the S3 bucket and to_localyzer/from_localyzer directories		* Set the S3 bucket and to_localyzer/from_localyzer directories
	S3_TO_LOCALYZER=s3://<S3 URL>/to_localyzer/		S3_TO_LOCALYZER=s3://<S3 URL>/to_localyzer/
	S3_FROM_LOCALYZER=s3://<S3 URL>/from_localyzer/		S3_FROM_LOCALYZER=s3://<S3 URL>/from_localyzer/
	** Set the DATA_DIR location for the working directories		* Set the DATA_DIR location for the working directories
	DATA_DIR="/full/path/git_s3_data"		DATA_DIR="/full/path/git_s3_data"
	** Set the commit message for the translated files going back to the repository:		* Set the commit message for the translated files going back to the repository:
	COMMIT_MESSAGE="Localyzer Git-S3"		COMMIT_MESSAGE="Localyzer Git-S3"


	* ''repositories.txt~~'':~~ Set the Git URL, branch, and optionally the directories to include, one per line, in the following format:		===repositories.txt===
			* Set the Git URL, branch, and optionally the directories to include, one per line, in the following format:
	https://<giturl>/<organization>/<repository> <branch> <optionally, comma separated list of include dirs>		https://<giturl>/<organization>/<repository> <branch> <optionally, comma separated list of include dirs>

Line 146:		Line 147:


	* ''fileSuffixes.txt~~'':~~ Set the resource file extensions so only those files are copied to the S3 bucket. For instance:		===fileSuffixes.txt===
			* Set the resource file extensions so only those files are copied to the S3 bucket. For instance:

	.properties		.properties

Olibouban: /* Verification */

2022-08-24T16:55:06Z

Verification

← Older revision		Revision as of 16:55, 24 August 2022
Line 155:		Line 155:
	To check that the '''Git to S3''' system is set up and configured properly:		To check that the '''Git to S3''' system is set up and configured properly:
	* make sure you do have at least one repository configured in the ''repositories.txt'' file		* make sure you do have at least one repository configured in the ''repositories.txt'' file
	* run the ~~git_to_s3~~.sh script		* run the <code>wrapper.sh</code> script
	* Look in the S3 bucket and verify the files of interest are there with <code>aws s3 ls</code> commands		* Look in the S3 bucket and verify the files of interest are there with <code>aws s3 ls</code> commands