Globalyzer Lite CICD Integration

From Lingoport Wiki
Revision as of 17:32, 10 May 2019 by Rdaly (talk | contribs) (→‎Overview)
Jump to: navigation, search

Instructions for integrating Globalyzer Scans into a CICD system.

Overview

CI/CD is continuous integration and continuous delivery. "Continuous integration is a coding philosophy and set of practices that drive development teams to implement small changes and check in code to version control repositories frequently. Because most modern applications require developing code in different platforms and tools, the team needs a mechanism to integrate and validate its changes." (InfoWorld)

Lingoport's Globalyzer can be used to scan software source code for internationalization issues.

Lingoport provides specific integration for GitHub pull request analysis. We have also provided ALM (Application Lifetime Management) integrations in the past.

For custom integrations with a CICD system, Globalyzer can be run against a specified list of files. The results of the code scan will then be placed in an XML report which can be parsed, with the results added as part of the CICD pipeline.

Running a custom pipeline

To integrate Globalyzer into a pipeline, use the Globalyzer Lite client application.

Globalyzer Lite is a java .jar application, which runs on java 1.8.

To run Globalyzer Lite, you will need a Project Defintion File with configuration settings.

You may optionally specify a comma-separated list of files to scan in order to narrow the analysis.

By default, Globalyzer Lite will connect with a Globalyzer Server to retrieve scanning configurations ("Rule Sets") and licensing information. The Rule Sets + License may alternately be downloaded and placed directly on the CICD system. This negates the need for network connectivity for the server. Note that if using locally downloaded Rule Sets + License, you will want to occasionally update both in order to keep up to date.

Globalyzer may use Machine Learning models in addition to the configured rule sets. If the models have been configured, they will need to be locally accessible for Globalyzer Lite to use them.

Detailed Documentation

Input

Files

  • Project Definition File
  • Optional:
    • Avoid Networking requirements to Globalyzer Server:
      • Local Rule Sets
      • Local License
    • Provide additional result classification
      • Machine Learning Model
    • General User Settings
      • .globalyzerrc

Command

Basic Example (full scan of source):

export WORKSPACE=<base directory of code to scan>
java -jar globalyzer-lite.jar -f "${WORKSPACE}/LiteProjectDefinition.xml" --project-path "${WORKSPACE}" --report-path "globalyzer-lite-reports"


Configurations

Additional useful configurations:

Specify files

--scan-items "comma,separate,list,of,paths,and,or,files" # Either Relative or Absolute are ok
export WORKSPACE=<base directory of code to scan>
java -jar globalyzer-lite.jar -f "${WORKSPACE}/LiteProjectDefinition.xml" --project-path "${WORKSPACE}" --report-path "globalyzer-lite-reports" --scan-items "foo/bar/,foo/baz/buzz.js,foo/baz/buzz2.js,foo/baz/beep.html"

Output

For CICD automations, the ScanDetailedXML report is recommended. Here is an abbreviated report (values changed to accurately reflect abbreviation):

Params

Non-obvious parameter specifications 

dfp=""  # Is this marked as a false positive in the Lingoport Dashboard?
pattern=""  # Regular Expression Pattern that matched for the detection
predicted=""  # count of Machine Learning predicted issues (if machine learning not used, default to all)
priority=""  # 0 - 5. 0 indicates string concatenation. 1 is most likely to be a real issue. 5 is most likely to be a false positive.
reason=""  # Reason for the detection or filtering of the issue
ruleSetName=""  # Rule Set that generated this section of results
scanName=""  # Specified in Project Definition File. Customizable.
usedLocalRuleset=""  # Did the scan happen with a local version of the Rule Set. If False, connection to Globalyzer Server used.
usedMachineLearning=""  # Were Machine Learning models applied?



<?xml version="1.0" encoding="UTF-8"?>
<GlobalyzerResults clientVersion="6.2.0_18" projectName="example" basepath="/var/lib/jenkins/jobs/ACME.example/workspace" companyName="ACME" endDate="2020-03-06" numLines="0" numProjects="0" numProducts="0" numRepositories="0">
  <Scan scanName="objC" ruleSetName="ACME_objC" language="objectivec" usedLocalRuleset="true" usedMachineLearning="false" lastScanDate="2019-04-11 16:22:36" filecount="3" linecount="4308" total="15" active="13" invalid="0" ignore="0" resolved="0" todo="0" filtered="2" dfp="0">
    <ScanSummary type="Embedded Strings" status="active" priority="all" prediction="all" count="2" predicted="2" />
    <ScanSummary type="Embedded Strings" status="invalid" count="0" />
    <ScanSummary type="Embedded Strings" status="ignore" count="0" />
    <ScanSummary type="Embedded Strings" status="resolved" count="0" />
    <ScanSummary type="Embedded Strings" status="todo" count="0" />
    <ScanSummary type="Embedded Strings" status="filtered" count="2" />
    <ScanSummary type="Embedded Strings" status="dfp" count="0" />
    <ScanResults type="Embedded Strings" status="active" priority="all" prediction="all" count="2" predicted="2">
      <result file="Apps/foo/bar/bazz.m" linenum="38" prediction="-" reason="String" priority="5" help="" description="" startpos="17" endpos="37">
        <issue>@"Camera Playground"</issue>
        <line>self.title = @"Camera Playground";</line>
      </result>
      <result file="Apps/foo/buzz/bizz.m" linenum="138" prediction="-" reason="String" priority="1" help="" description="" startpos="30" endpos="61">
        <issue>@"Gain valuable brand insights"</issue>
        <line>titleLabel.text = @"Gain valuable brand insights";</line>
      </result>
    </ScanResults>
    <ScanResults type="Embedded Strings" status="invalid" count="0" />
    <ScanResults type="Embedded Strings" status="ignore" count="0" />
    <ScanResults type="Embedded Strings" status="resolved" count="0" />
    <ScanResults type="Embedded Strings" status="todo" count="0" />
    <ScanResults type="Embedded Strings" status="filtered" count="2">
      <result file="Apps/foo/bar/bazz.m" linenum="8" prediction="Negative" reason="No dictionary words" priority="" help="" description="" startpos="8" endpos="39">
        <issue>"CameraDelegate.h"</issue>
        <line>#import "CameraDelegate.h"</line>
      </result>
      <result file="Apps/foo/buzz/bizz.m" linenum="10" prediction="Negative" reason="No dictionary words" priority="" help="" description="" startpos="8" endpos="36">
        <issue>"CameraLauncher.h"</issue>
        <line>#import "CameraLauncher.h"</line>
      </result>
    </ScanResults>
    <ScanResults type="Embedded Strings" status="dfp" count="0" />
    <ScanSummary type="Locale-Sensitive Methods" status="active" priority="all" prediction="all" count="4" predicted="4">
      <ScanSummaryPattern pattern="\bdate\b" count="1" help="https://www.globalyzer.com/gzserver/help/reference/localeSensitiveMethods/objC_date.html" />
      <ScanSummaryPattern pattern="boldSystemFontOfSize" count="1" help="https://www.globalyzer.com/gzserver/help/reference/localeSensitiveMethods/objC_boldSystemFontOfSize.html" />
      <ScanSummaryPattern pattern="stringWithFormat" count="1" help="https://www.globalyzer.com/gzserver/help/reference/localeSensitiveMethods/objC_stringWithFormat.html" />
      <ScanSummaryPattern pattern="whitespaceAndNewlineCharacterSet" count="1" help="https://www.globalyzer.com/gzserver/help/reference/localeSensitiveMethods/objC_whitespaceAndNewlineCharacterSet.html" />
    </ScanSummary>
    <ScanSummary type="Locale-Sensitive Methods" status="invalid" count="0" />
    <ScanSummary type="Locale-Sensitive Methods" status="ignore" count="0" />
    <ScanSummary type="Locale-Sensitive Methods" status="resolved" count="0" />
    <ScanSummary type="Locale-Sensitive Methods" status="todo" count="0" />
    <ScanSummary type="Locale-Sensitive Methods" status="filtered" count="0" />
    <ScanSummary type="Locale-Sensitive Methods" status="dfp" count="0" />
    <ScanResults type="Locale-Sensitive Methods" status="active" priority="all" prediction="all" count="4" predicted="4">
      <result file="Apps/foo/bar/bazz.m" linenum="125" prediction="-" reason="String Formatting Operations:boldSystemFontOfSize" priority="2" help="https://www.globalyzer.com/gzserver/help/reference/localeSensitiveMethods/objC_boldSystemFontOfSize.html" description="font features" startpos="34" endpos="54">
        <issue>boldSystemFontOfSize</issue>
        <line>titleLabel.font = [UIFont boldSystemFontOfSize:22];</line>
      </result>
      <result file="Apps/foo/buzz/bizz.m" linenum="67" prediction="-" reason="String Formatting Operations:stringWithFormat" priority="2" help="https://www.globalyzer.com/gzserver/help/reference/localeSensitiveMethods/objC_stringWithFormat.html" description="NSString stringWithFormat" startpos="33" endpos="49">
        <issue>stringWithFormat</issue>
        <line>[self updateStatus:[NSString stringWithFormat:format, error]];</line>
      </result>
      <result file="Apps/foo/buzz/bizz.m" linenum="293" prediction="-" reason="Date/Time:\bdate\b" priority="1" help="https://www.globalyzer.com/gzserver/help/reference/localeSensitiveMethods/objC_date.html" description="NSDateComponents date" startpos="30" endpos="34">
        <issue>date</issue>
        <line>long msUtc1970 = [[NSDate date] timeIntervalSince1970] * 1000;</line>
      </result>
      <result file="Apps/foo/buzz/bizz.m" linenum="308" prediction="-" reason="String Formatting Operations:whitespaceAndNewlineCharacterSet" priority="2" help="https://www.globalyzer.com/gzserver/help/reference/localeSensitiveMethods/objC_whitespaceAndNewlineCharacterSet.html" description="NSCharacterSet whitespaceAndNewlineCharacterSet" startpos="77" endpos="109">
        <issue>whitespaceAndNewlineCharacterSet</issue>
        <line>hostURLStr = [hostURLStr stringByTrimmingCharactersInSet:[NSCharacterSet whitespaceAndNewlineCharacterSet]];</line>
      </result>
    </ScanResults>
    <ScanResults type="Locale-Sensitive Methods" status="invalid" count="0" />
    <ScanResults type="Locale-Sensitive Methods" status="ignore" count="0" />
    <ScanResults type="Locale-Sensitive Methods" status="resolved" count="0" />
    <ScanResults type="Locale-Sensitive Methods" status="todo" count="0" />
    <ScanResults type="Locale-Sensitive Methods" status="filtered" count="0" />
    <ScanResults type="Locale-Sensitive Methods" status="dfp" count="0" />
    <ScanSummary type="General Patterns" status="active" priority="all" prediction="all" count="4" predicted="3">
      <ScanSummaryPattern pattern="append" count="1" help="" />
      <ScanSummaryPattern pattern="NSCharacterSet" count="1" help="" />
      <ScanSummaryPattern pattern="NSDate\b" count="1" help="https://developer.apple.com/library/prerelease/mac/documentation/Cocoa/Reference/Foundation/Classes/NSDate_Class/" />
      <ScanSummaryPattern pattern="NSNumericSearch" count="1" help="" />
    </ScanSummary>
    <ScanSummary type="General Patterns" status="invalid" count="0" />
    <ScanSummary type="General Patterns" status="ignore" count="0" />
    <ScanSummary type="General Patterns" status="resolved" count="0" />
    <ScanSummary type="General Patterns" status="todo" count="0" />
    <ScanSummary type="General Patterns" status="filtered" count="0" />
    <ScanSummary type="General Patterns" status="dfp" count="0" />
    <ScanResults type="General Patterns" status="active" priority="all" prediction="all" count="4" predicted="3">
      <result file="Apps/foo/bar/bazz.m" linenum="190" prediction="-" reason="NSDate\b" priority="4" help="https://developer.apple.com/library/prerelease/mac/documentation/Cocoa/Reference/Foundation/Classes/NSDate_Class/" description="Date object" startpos="34" endpos="40">
        <issue>NSDate</issue>
        <line>captureTimeStamp:(NSDate *)captureTimeStamp</line>
      </result>
      <result file="Apps/foo/bar/bazz.m" linenum="290" prediction="-" reason="append\b" priority="0" help="" description="String Concatenation" startpos="54" endpos="80">
        <issue>append</issue>
        <line>uiText.append(description)</line>
      </result>
      <result file="Apps/foo/bar/bazz.m" linenum="308" prediction="-" reason="NSCharacterSet" priority="4" help="" description="NSCharacterSet" startpos="62" endpos="76">
        <issue>NSCharacterSet</issue>
        <line>hostURLStr = [hostURLStr stringByTrimmingCharactersInSet:[NSCharacterSet whitespaceAndNewlineCharacterSet]];</line>
      </result>
      <result file="Apps/foo/buzz/bizz.m" linenum="347" prediction="-" reason="NSNumericSearch" priority="2" help="" description="NSString Search options" startpos="52" endpos="67">
        <issue>NSNumericSearch</issue>
        <line>if ([remoteVersion compare:localVersion options:NSNumericSearch] == NSOrderedDescending) {</line>
      </result>
    </ScanResults>
    <ScanResults type="General Patterns" status="invalid" count="0" />
    <ScanResults type="General Patterns" status="ignore" count="0" />
    <ScanResults type="General Patterns" status="resolved" count="0" />
    <ScanResults type="General Patterns" status="todo" count="0" />
    <ScanResults type="General Patterns" status="filtered" count="0" />
    <ScanResults type="General Patterns" status="dfp" count="0" />
    <ScanSummary type="Static File References" status="active" priority="all" prediction="all" count="3" predicted="3">
      <ScanSummaryPattern pattern="[\w\-\.]+\.jpe?g\b(?!\.)(?!(\s*\())" count="2" help="" />
      <ScanSummaryPattern pattern="[\w\-\.]+\.png\b(?!\.)(?!(\s*\())" count="1" help="" />
    </ScanSummary>
    <ScanSummary type="Static File References" status="invalid" count="0" />
    <ScanSummary type="Static File References" status="ignore" count="0" />
    <ScanSummary type="Static File References" status="resolved" count="0" />
    <ScanSummary type="Static File References" status="todo" count="0" />
    <ScanSummary type="Static File References" status="filtered" count="0" />
    <ScanSummary type="Static File References" status="dfp" count="0" />
    <ScanResults type="Static File References" status="active" priority="all" prediction="all" count="3" predicted="3">
      <result file="Apps/foo/bar/bazz.m" linenum="168" prediction="-" reason="Static File" priority="1" help="" description="Image" startpos="94" endpos="116">
        <issue>Back_to_the_Future.jpg</issue>
        <line>@"thumbnail" : @"https://upload.wikimedia.org/wikipedia/en/d/d2/Back_to_the_Future.jpg",</line>
      </result>
      <result file="Apps/foo/bar/bazz.m" linenum="387" prediction="-" reason="Static File" priority="1" help="" description="Image" startpos="83" endpos="102">
        <issue>logo_primary.png</issue>
        <line>@"https://www.someurl.com/img2/branding/logo_primary.png",</line>
      </result>
      <result file="Apps/foo/buzz/bizz.m" linenum="431" prediction="-" reason="Static File" priority="1" help="" description="Image" startpos="72" endpos="113">
        <issue>Music-Star.jpg</issue>
        <line>@"https://www.otherurl.com/d/l400/pict/123456789_/Music-Star.jpg",</line>
      </result>
    </ScanResults>
    <ScanResults type="Static File References" status="invalid" count="0" />
    <ScanResults type="Static File References" status="ignore" count="0" />
    <ScanResults type="Static File References" status="resolved" count="0" />
    <ScanResults type="Static File References" status="todo" count="0" />
    <ScanResults type="Static File References" status="filtered" count="0" />
    <ScanResults type="Static File References" status="dfp" count="0" />
  </Scan>
</GlobalyzerResults>

Retrieved from "http://wiki.lingoport.com/index.php?title=Globalyzer_Lite_CICD_Integration&oldid=92901"