Difference between revisions of "Globalyzer Lite CICD Integration"

From Lingoport Wiki
Jump to: navigation, search
(→‎Configurations)
(→‎Configurations)
Line 56: Line 56:
 
: <code>--scan-items "comma,separate,list,of,paths,and,or,files" # Either Relative or Absolute are ok
  
: <code>--scan-items "comma,separate,list,of,paths,and,or,files" # Either Relative or Absolute are ok
   
  +
<pre>
−
: export WORKSPACE=<base directory of code to scan>
  
  +
export WORKSPACE=<base directory of code to scan>
−  
−
: <code>java -jar globalyzer-lite.jar -f "${WORKSPACE}/LiteProjectDefinition.xml" --project-path "${WORKSPACE}" --report-path "globalyzer-lite-reports" --scan-items "foo/bar/,foo/baz/buzz.js,foo/baz/buzz2.js,foo/baz/beep.html"</code>
 +
java -jar globalyzer-lite.jar -f "${WORKSPACE}/LiteProjectDefinition.xml" --project-path "${WORKSPACE}" --report-path "globalyzer-lite-reports" --scan-items "foo/bar/,foo/baz/buzz.js,foo/baz/buzz2.js,foo/baz/beep.html"
  +
</pre>
   
 
== Output ==
  
== Output ==

Revision as of 18:57, 10 May 2019

Instructions for integrating Globalyzer Scans into a CICD system.

Overview

CI/CD is continuous integration and continuous delivery. "Continuous integration is a coding philosophy and set of practices that drive development teams to implement small changes and check in code to version control repositories frequently. Because most modern applications require developing code in different platforms and tools, the team needs a mechanism to integrate and validate its changes." (InfoWorld)

Lingoport's Globalyzer can be used to scan software source code for internationalization issues.

Lingoport provides specific integration for GitHub pull request analysis. We have also provided ALM (Application Lifetime Management) integrations in the past.

For custom integrations with a CI/CD system, Globalyzer can be run against a specified list of files. The results of the code scan will then be placed in an XML report which can be parsed, with the results added as part of the CI/CD pipeline.

Running a custom pipeline

To integrate Globalyzer into a pipeline, use the Globalyzer Lite application.

To run Globalyzer Lite, you will need a Project Defintion File with configuration settings. The Project Definition File allows the user to specify a comma-separated list of files to scan in order to narrow the analysis or scan a whole project.

By default, Globalyzer Lite will connect with a Globalyzer Server to retrieve scanning configurations ("Rule Sets") and licensing information. The Rule Sets and the License may also be downloaded and placed locally on the CI/CD system. This negates the need for network connectivity to the server. Note that if using locally downloaded Rule Sets and licenses, you will want to occasionally update both in order to keep up to date.

Globalyzer may use Machine Learning models in addition to the configured rule sets. If the models have been configured, they will need to be locally accessible for Globalyzer Lite to use them.

Detailed Documentation

Input

Files

  • Project Definition File
  • Optional:
    • Avoid Networking requirements to Globalyzer Server:
      • Local Rule Sets
      • Local License
    • Provide additional result classification
      • Machine Learning Model
    • General User Settings
      • .globalyzerrc

Command

Basic Example (full scan of source):

export WORKSPACE=<base directory of code to scan>
java -jar globalyzer-lite.jar -f "${WORKSPACE}/LiteProjectDefinition.xml" --project-path "${WORKSPACE}" --report-path "globalyzer-lite-reports"


Configurations

Additional useful configurations:

Specify files

--scan-items "comma,separate,list,of,paths,and,or,files" # Either Relative or Absolute are ok
export WORKSPACE=<base directory of code to scan>
java -jar globalyzer-lite.jar -f "${WORKSPACE}/LiteProjectDefinition.xml" --project-path "${WORKSPACE}" --report-path "globalyzer-lite-reports" --scan-items "foo/bar/,foo/baz/buzz.js,foo/baz/buzz2.js,foo/baz/beep.html"

Output

For CI/CD automations, the ScanDetailedXML report is recommended. Here is an abbreviated report (values changed to accurately reflect abbreviation):

Parameters

Definitions of some of the parameters used in the report 

dfp=""  # Is this marked as a false positive in the Lingoport Dashboard?
pattern=""  # Regular Expression Pattern that matched for the detection
predicted=""  # count of Machine Learning predicted issues (if machine learning is not used, default to all)
priority=""  # Severity of the issue. 0 - 5. 0 indicates string concatenation. 1 is most likely to be a real issue. 5 is most likely to be a false positive.
reason=""  # Reason for the detection or filtering of the issue
ruleSetName=""  # Rule Set that generated this section of results
scanName=""  # Specified in Project Definition File. Customizable.
usedLocalRuleset=""  # Did the scan happen with a local version of the Rule Set? If False, connection to Globalyzer Server was used.
usedMachineLearning=""  # Were Machine Learning models applied?

ScanDetailedXML report 

<?xml version="1.0" encoding="UTF-8"?>
<GlobalyzerResults clientVersion="6.2.0_18" projectName="example" basepath="/var/lib/jenkins/jobs/ACME.example/workspace" companyName="ACME" endDate="2020-03-06" numLines="0" numProjects="0" numProducts="0" numRepositories="0">
  <Scan scanName="objC" ruleSetName="ACME_objC" language="objectivec" usedLocalRuleset="true" usedMachineLearning="false" lastScanDate="2019-04-11 16:22:36" filecount="3" linecount="4308" total="15" active="13" invalid="0" ignore="0" resolved="0" todo="0" filtered="2" dfp="0">
    <ScanSummary type="Embedded Strings" status="active" priority="all" prediction="all" count="2" predicted="2" />
    <ScanSummary type="Embedded Strings" status="invalid" count="0" />
    <ScanSummary type="Embedded Strings" status="ignore" count="0" />
    <ScanSummary type="Embedded Strings" status="resolved" count="0" />
    <ScanSummary type="Embedded Strings" status="todo" count="0" />
    <ScanSummary type="Embedded Strings" status="filtered" count="2" />
    <ScanSummary type="Embedded Strings" status="dfp" count="0" />
    <ScanResults type="Embedded Strings" status="active" priority="all" prediction="all" count="2" predicted="2">
      <result file="Apps/foo/bar/bazz.m" linenum="38" prediction="-" reason="String" priority="5" help="" description="" startpos="17" endpos="37">
        <issue>@"Camera Playground"</issue>
        <line>self.title = @"Camera Playground";</line>
      </result>
      <result file="Apps/foo/buzz/bizz.m" linenum="138" prediction="-" reason="String" priority="1" help="" description="" startpos="30" endpos="61">
        <issue>@"Gain valuable brand insights"</issue>
        <line>titleLabel.text = @"Gain valuable brand insights";</line>
      </result>
    </ScanResults>
    <ScanResults type="Embedded Strings" status="invalid" count="0" />
    <ScanResults type="Embedded Strings" status="ignore" count="0" />
    <ScanResults type="Embedded Strings" status="resolved" count="0" />
    <ScanResults type="Embedded Strings" status="todo" count="0" />
    <ScanResults type="Embedded Strings" status="filtered" count="2">
      <result file="Apps/foo/bar/bazz.m" linenum="8" prediction="Negative" reason="No dictionary words" priority="" help="" description="" startpos="8" endpos="39">
        <issue>"CameraDelegate.h"</issue>
        <line>#import "CameraDelegate.h"</line>
      </result>
      <result file="Apps/foo/buzz/bizz.m" linenum="10" prediction="Negative" reason="No dictionary words" priority="" help="" description="" startpos="8" endpos="36">
        <issue>"CameraLauncher.h"</issue>
        <line>#import "CameraLauncher.h"</line>
      </result>
    </ScanResults>
    <ScanResults type="Embedded Strings" status="dfp" count="0" />
    <ScanSummary type="Locale-Sensitive Methods" status="active" priority="all" prediction="all" count="4" predicted="4">
      <ScanSummaryPattern pattern="\bdate\b" count="1" help="https://www.globalyzer.com/gzserver/help/reference/localeSensitiveMethods/objC_date.html" />
      <ScanSummaryPattern pattern="boldSystemFontOfSize" count="1" help="https://www.globalyzer.com/gzserver/help/reference/localeSensitiveMethods/objC_boldSystemFontOfSize.html" />
      <ScanSummaryPattern pattern="stringWithFormat" count="1" help="https://www.globalyzer.com/gzserver/help/reference/localeSensitiveMethods/objC_stringWithFormat.html" />
      <ScanSummaryPattern pattern="whitespaceAndNewlineCharacterSet" count="1" help="https://www.globalyzer.com/gzserver/help/reference/localeSensitiveMethods/objC_whitespaceAndNewlineCharacterSet.html" />
    </ScanSummary>
    <ScanSummary type="Locale-Sensitive Methods" status="invalid" count="0" />
    <ScanSummary type="Locale-Sensitive Methods" status="ignore" count="0" />
    <ScanSummary type="Locale-Sensitive Methods" status="resolved" count="0" />
    <ScanSummary type="Locale-Sensitive Methods" status="todo" count="0" />
    <ScanSummary type="Locale-Sensitive Methods" status="filtered" count="0" />
    <ScanSummary type="Locale-Sensitive Methods" status="dfp" count="0" />
    <ScanResults type="Locale-Sensitive Methods" status="active" priority="all" prediction="all" count="4" predicted="4">
      <result file="Apps/foo/bar/bazz.m" linenum="125" prediction="-" reason="String Formatting Operations:boldSystemFontOfSize" priority="2" help="https://www.globalyzer.com/gzserver/help/reference/localeSensitiveMethods/objC_boldSystemFontOfSize.html" description="font features" startpos="34" endpos="54">
        <issue>boldSystemFontOfSize</issue>
        <line>titleLabel.font = [UIFont boldSystemFontOfSize:22];</line>
      </result>
      <result file="Apps/foo/buzz/bizz.m" linenum="67" prediction="-" reason="String Formatting Operations:stringWithFormat" priority="2" help="https://www.globalyzer.com/gzserver/help/reference/localeSensitiveMethods/objC_stringWithFormat.html" description="NSString stringWithFormat" startpos="33" endpos="49">
        <issue>stringWithFormat</issue>
        <line>[self updateStatus:[NSString stringWithFormat:format, error]];</line>
      </result>
      <result file="Apps/foo/buzz/bizz.m" linenum="293" prediction="-" reason="Date/Time:\bdate\b" priority="1" help="https://www.globalyzer.com/gzserver/help/reference/localeSensitiveMethods/objC_date.html" description="NSDateComponents date" startpos="30" endpos="34">
        <issue>date</issue>
        <line>long msUtc1970 = [[NSDate date] timeIntervalSince1970] * 1000;</line>
      </result>
      <result file="Apps/foo/buzz/bizz.m" linenum="308" prediction="-" reason="String Formatting Operations:whitespaceAndNewlineCharacterSet" priority="2" help="https://www.globalyzer.com/gzserver/help/reference/localeSensitiveMethods/objC_whitespaceAndNewlineCharacterSet.html" description="NSCharacterSet whitespaceAndNewlineCharacterSet" startpos="77" endpos="109">
        <issue>whitespaceAndNewlineCharacterSet</issue>
        <line>hostURLStr = [hostURLStr stringByTrimmingCharactersInSet:[NSCharacterSet whitespaceAndNewlineCharacterSet]];</line>
      </result>
    </ScanResults>
    <ScanResults type="Locale-Sensitive Methods" status="invalid" count="0" />
    <ScanResults type="Locale-Sensitive Methods" status="ignore" count="0" />
    <ScanResults type="Locale-Sensitive Methods" status="resolved" count="0" />
    <ScanResults type="Locale-Sensitive Methods" status="todo" count="0" />
    <ScanResults type="Locale-Sensitive Methods" status="filtered" count="0" />
    <ScanResults type="Locale-Sensitive Methods" status="dfp" count="0" />
    <ScanSummary type="General Patterns" status="active" priority="all" prediction="all" count="4" predicted="3">
      <ScanSummaryPattern pattern="append" count="1" help="" />
      <ScanSummaryPattern pattern="NSCharacterSet" count="1" help="" />
      <ScanSummaryPattern pattern="NSDate\b" count="1" help="https://developer.apple.com/library/prerelease/mac/documentation/Cocoa/Reference/Foundation/Classes/NSDate_Class/" />
      <ScanSummaryPattern pattern="NSNumericSearch" count="1" help="" />
    </ScanSummary>
    <ScanSummary type="General Patterns" status="invalid" count="0" />
    <ScanSummary type="General Patterns" status="ignore" count="0" />
    <ScanSummary type="General Patterns" status="resolved" count="0" />
    <ScanSummary type="General Patterns" status="todo" count="0" />
    <ScanSummary type="General Patterns" status="filtered" count="0" />
    <ScanSummary type="General Patterns" status="dfp" count="0" />
    <ScanResults type="General Patterns" status="active" priority="all" prediction="all" count="4" predicted="3">
      <result file="Apps/foo/bar/bazz.m" linenum="190" prediction="-" reason="NSDate\b" priority="4" help="https://developer.apple.com/library/prerelease/mac/documentation/Cocoa/Reference/Foundation/Classes/NSDate_Class/" description="Date object" startpos="34" endpos="40">
        <issue>NSDate</issue>
        <line>captureTimeStamp:(NSDate *)captureTimeStamp</line>
      </result>
      <result file="Apps/foo/bar/bazz.m" linenum="290" prediction="-" reason="append\b" priority="0" help="" description="String Concatenation" startpos="54" endpos="80">
        <issue>append</issue>
        <line>uiText.append(description)</line>
      </result>
      <result file="Apps/foo/bar/bazz.m" linenum="308" prediction="-" reason="NSCharacterSet" priority="4" help="" description="NSCharacterSet" startpos="62" endpos="76">
        <issue>NSCharacterSet</issue>
        <line>hostURLStr = [hostURLStr stringByTrimmingCharactersInSet:[NSCharacterSet whitespaceAndNewlineCharacterSet]];</line>
      </result>
      <result file="Apps/foo/buzz/bizz.m" linenum="347" prediction="-" reason="NSNumericSearch" priority="2" help="" description="NSString Search options" startpos="52" endpos="67">
        <issue>NSNumericSearch</issue>
        <line>if ([remoteVersion compare:localVersion options:NSNumericSearch] == NSOrderedDescending) {</line>
      </result>
    </ScanResults>
    <ScanResults type="General Patterns" status="invalid" count="0" />
    <ScanResults type="General Patterns" status="ignore" count="0" />
    <ScanResults type="General Patterns" status="resolved" count="0" />
    <ScanResults type="General Patterns" status="todo" count="0" />
    <ScanResults type="General Patterns" status="filtered" count="0" />
    <ScanResults type="General Patterns" status="dfp" count="0" />
    <ScanSummary type="Static File References" status="active" priority="all" prediction="all" count="3" predicted="3">
      <ScanSummaryPattern pattern="[\w\-\.]+\.jpe?g\b(?!\.)(?!(\s*\())" count="2" help="" />
      <ScanSummaryPattern pattern="[\w\-\.]+\.png\b(?!\.)(?!(\s*\())" count="1" help="" />
    </ScanSummary>
    <ScanSummary type="Static File References" status="invalid" count="0" />
    <ScanSummary type="Static File References" status="ignore" count="0" />
    <ScanSummary type="Static File References" status="resolved" count="0" />
    <ScanSummary type="Static File References" status="todo" count="0" />
    <ScanSummary type="Static File References" status="filtered" count="0" />
    <ScanSummary type="Static File References" status="dfp" count="0" />
    <ScanResults type="Static File References" status="active" priority="all" prediction="all" count="3" predicted="3">
      <result file="Apps/foo/bar/bazz.m" linenum="168" prediction="-" reason="Static File" priority="1" help="" description="Image" startpos="94" endpos="116">
        <issue>Back_to_the_Future.jpg</issue>
        <line>@"thumbnail" : @"https://upload.wikimedia.org/wikipedia/en/d/d2/Back_to_the_Future.jpg",</line>
      </result>
      <result file="Apps/foo/bar/bazz.m" linenum="387" prediction="-" reason="Static File" priority="1" help="" description="Image" startpos="83" endpos="102">
        <issue>logo_primary.png</issue>
        <line>@"https://www.someurl.com/img2/branding/logo_primary.png",</line>
      </result>
      <result file="Apps/foo/buzz/bizz.m" linenum="431" prediction="-" reason="Static File" priority="1" help="" description="Image" startpos="72" endpos="113">
        <issue>Music-Star.jpg</issue>
        <line>@"https://www.otherurl.com/d/l400/pict/123456789_/Music-Star.jpg",</line>
      </result>
    </ScanResults>
    <ScanResults type="Static File References" status="invalid" count="0" />
    <ScanResults type="Static File References" status="ignore" count="0" />
    <ScanResults type="Static File References" status="resolved" count="0" />
    <ScanResults type="Static File References" status="todo" count="0" />
    <ScanResults type="Static File References" status="filtered" count="0" />
    <ScanResults type="Static File References" status="dfp" count="0" />
  </Scan>
</GlobalyzerResults>


Some details in the report: 

 <Scan scanName="objC" ruleSetName="ACME_objC" language="objectivec" usedLocalRuleset="true" usedMachineLearning="false" lastScanDate="2019-04-11 16:22:36" filecount="3" linecount="4308" total="15" active="13" invalid="0" ignore="0" resolved="0" todo="0" filtered="2" dfp="0">
  • The name of the scan is "objC"
  • The ruleset used is "ACME_objC"
  • This is a ruleset for the ObjectiveC language
  • The ruleset was local to the system
  • Machine learning was not used.
  • There were 3 files scanned that had a total of 4308 lines
  • 15 issues were found in the scan and 13 of them are active issues and 2 were filtered out.
<ScanSummary type="Embedded Strings" status="active" priority="all" prediction="all" count="2" predicted="2" />
<ScanSummary type="Locale-Sensitive Methods" status="active" priority="all" prediction="all" count="4" predicted="4">
<ScanSummary type="General Patterns" status="active" priority="all" prediction="all" count="4" predicted="3">
<ScanResults type="Static File References" status="active" priority="all" prediction="all" count="3" predicted="3">

Each type of issue, Embedded Strings, Locale-Sensitive Methods, General Patterns and Static File References, had issues to report. 2,4,4, and 3 issues respectively for a total of 13 active issues as denoted in the <Scan stanza. The issues are detailed in their respective area.

Retrieved from "http://wiki.lingoport.com/index.php?title=Globalyzer_Lite_CICD_Integration&oldid=92916"