Difference between revisions of "False Positives"

From Lingoport Wiki
Jump to: navigation, search
(Modifying Rules)
 
(43 intermediate revisions by 4 users not shown)
Line 1: Line 1:
=What are False Positives=
+
=What are False Positives?=
 
Globalyzer scans the source code to detect candidate issues based on rules. Given some code, a rule may detect a candidate issue. In one context, that candidate issue may need to be corrected when in another context, the same issue may not need any work and is a false positive.
 
Globalyzer scans the source code to detect candidate issues based on rules. Given some code, a rule may detect a candidate issue. In one context, that candidate issue may need to be corrected when in another context, the same issue may not need any work and is a false positive.
   
Let's look at a couple of lines of Java code which creates a date string based on a hard coded date format using <code>SimpleDateFormat</code>:
+
Let's look at a couple of lines of Java code which creates a date string based on a hard coded date format using '''<code>SimpleDateFormat</code>:'''
   
SimpleDateFormat formatter =new SimpleDateFormat("MM/dd/yy");
+
SimpleDateFormat formatter = new SimpleDateFormat("MM/dd/yy");
 
String dateString = formatter.format(new Date());
 
String dateString = formatter.format(new Date());
   
Using the default Java rule sets, this code will raise i18n issues. If the <code>dateString</code> variable is user facing, this code is indeed an issue to be fixed (for more information, check your rule set help pages).
+
Using the default Java rule set, this code will raise [[Terms_and_Definitions#i18n|i18n]] issues. If the <code>dateString</code> variable is user facing, this code is indeed an issue to be fixed (for more information, check your rule set help pages).
   
However, if the <code>dateString</code> variable is used for internal purposes, such as a in a support log file, it should not be modified and is a false positive.
+
However, if the <code>dateString</code> variable is used for internal purposes, such as in a support log file, it should not be modified and is a false positive.
   
 
This section describes different ways to handle false positives.
 
This section describes different ways to handle false positives.
  +
=Globalyzer Comment Tags=
  +
When you want to filter out a specific item in the code, one way is to add a Globalyzer comment on the offending line. This is for a specific item and only this one. If more code is written in the same way and Globalyzer should not detect the issue, a Rule Set modification may be better.
   
  +
Let's look at a simple line of code where Globalyzer detects a candidate issue, here another embedded string:
=Modifying Rules=
 
'''Note: For more detailed information and strategies on filtering rules, please visit the [[False_Positive_Filtering | False Positive Filtering]] page.'''
 
   
  +
String marker = "Marker 15";
Since Rules are what detect or filter out issues, the most common way to handle particular programming styles is to modify the rule set applied to scan a project's code.
 
   
Let's look at a string detected by Globalyzer:
 
   
  +
By adding the <b><code>//$NON-NLS-L$</code></b> comment at the end of the line, Globalyzer will now ignore the issue:
SpecialTB.write("New user created");
 
   
  +
String marker = "Marker 15"; //$NON-NLS-L$
   
Globalyzer identifies the string <code>New user created</code> as a candidate embedded string which should be moved out of the code and into a resource bundle. However, let's assume the developer knows that SpecialTB.write() is a method that writes some logging information into a database for potential debugging purposes. The Rule Set needs to be modified based on that knowledge so that this particular line and all other strings passed to this call will be filtered out, i.e. will not be detected as a candidate issue. In this instance, a String Method Filter would be created as shown below:
 
   
  +
Since the code is pushed to the source control system, the next time a Workbench user or a script gets the latest code, Globalyzer will ignore the issue.
[[File:Rule SpecialTB write.gif]]
 
   
  +
The default list of Globalyzer Tags is:
Any Globalyzer scan using this updated rule set will henceforth filter out the embedded string issue. Workbench instances and the nightly builds for instance would now filter out the issue.
 
  +
* <code>$NON-NLS-L$</code> : Ignores all issues on the line (all Embedded Strings, Locale-Sensitive Methods, General Patterns, and Static File References found on the line)
  +
* <code>$NON-NLS-[number]$</code> : This is for Embedded Strings only and ignores one particular string on the line. For instance $NON-NLS-2$ ignores the second Embedded String on the line
  +
* <code>GLOBALYZER_TODO</code> : Indicates something to be done in i18n. It overrides/detects the line and sets a TODO status to it.
  +
* <code>GLOBALYZER_IGNORE_NEXT_LINE</code>: Ignores any issue on the line after this comment
  +
* <code>GLOBALYZER_START_IGNORE</code> : Ignores a block of issues until the GLOBALYZER_END_IGNORE tag comment
  +
* <code>GLOBALYZER_END_IGNORE</code> : Stops ignoring a block of issues, working with GLOBALYZER_START_IGNORE
   
You can also create and try new rules from the Workbench, and push the new rules to the server. To do so, right mouse click on an issue and select the "Add Rule Sets Filters/Detection ..." to show the dialog box.
 
   
== More Precise Rules ==
+
=Modifying Rules=
  +
Since Rules are what detect or filter out issues, the most common way to handle particular programming styles is to modify the rule set applied to scan a project's code.
With Globalyzer 5.0, rules are moving to an AST based parsing which provide more powerful rules, starting with Java. Click [[Globalyzer 5 Java Rules|Globalyzer 5 Java Rules]] for more information.
 
   
  +
Let's look at a string detected by Globalyzer:
== Detailed Guide ==
 
   
  +
SpecialTB.write("New user created");
Visit the [False Positive Filtering] page for a detailed guide on creating false-positive filtering rules.
 
   
=Globalyzer Comment Tags=
 
When you want to filter out a specific item in the code, one way is to add a Globalyzer comment on the offending line. This is for a specific item and only this one. If more code is written in the same way and Globalyzer should not detect the issue, a Rule Set modification may be better.
 
   
  +
Globalyzer identifies the string <code>New user created</code> as a candidate embedded string which should be moved out of the code and into a resource file. However, let's assume the developer knows that SpecialTB.write() is a method that writes some logging information into a database for potential debugging purposes. The Rule Set needs to be modified based on that knowledge so that this particular line and all other strings passed to this call will be filtered out, i.e. will not be detected as a candidate issue. In this instance, a '''String Method Filter''' would be created as shown below:
Let's look at a simple line of code where Globalyzer detects a candidate issue, here another embedded string:
 
   
  +
<br>
String marker = "Marker 15";
 
  +
[[File:Add_string_method_filter.png]]
  +
<br>
   
  +
<b>Note:</b> for the Java Rule Set only, we support Class or Variable Types, so you can filter strings when
  +
passed as a parameter to a particular object/method.
  +
  +
Any Globalyzer scan using this updated rule set will henceforth filter out the embedded string issue. Workbench instances and the nightly builds for instance would now filter out the issue.
   
  +
You can also create and try new rules from the Globalyzer Workbench, and push the new rules to the server. To do so, right mouse click on an issue and select the '''Add Rule Sets Filters/Detection ...''' to show the dialog box.
By adding the <b><code>//$NON-NLS-L$</code></b> comment at the end of the line, Globalyzer will now ignore the issue:
 
   
String marker = "Marker 15"; //$NON-NLS-L$
 
   
  +
== Detailed Guide ==
   
  +
Please visit the [[False_Positive_Filtering | False Positive Filtering]] page for a detailed guide on creating false-positive filtering rules.
Since the code is pushed to the source control system, the next time a Workbench user or a script gets the latest code, Globalyzer will ignore the issue.
 
   
The default list of Globalyzer Tags is:
+
=Globalyzer Workbench Prediction=
  +
Predictions are used to differentiate true issues from false positive ones.
* <code>$NON-NLS-L$</code> : Ignores any issue on the line
 
  +
When using the Globalyzer Workbench, you may want to change the prediction of some issues locally to your instance of the Workbench.
* <code>$NON-NLS-[number]$</code> : Ignores one particular issue on the line. For instance $NON-NLS-2$ ignores the second issue on the line
 
  +
This will update the database tracking issues on your system; It does <b>not</b> affect what other users or system detect.
* <code>GLOBALYZER_TODO</code> : Indicates something to be done in i18n. It overrides/detects the line and sets a TODO status to it.
 
  +
This helps organize the issues locally quickly, for instance when externalizing strings with the Workbench. You can mark the prediction of issues as <code>TRUE(T)</code> if you
* <code>GLOBALYZER_IGNORE_NEXT_LINE</code>: Ignores any issue on the line after this comment
 
  +
know the issue is a true positive, <code>FALSE(F)</code> if you know the issue is a false positive, or <code>PENDING(P)</code> if you are unsure but you want to indicate
* <code>GLOBALYZER_START_IGNORE</code> : Ignores a block of issues until the GLOBALYZER_END_IGNORE tag comment
 
  +
that the issue needs further evaluation.
* <code>GLOBALYZER_END_IGNORE</code> : Stops ignoring a block of issues, working with GLOBALYZER_START_IGNORE
 
   
  +
To change the prediction of an issue, select it, click the right mouse button, and click on the menu item. In the example below, the prediction of the issue will be 'F' locally.
=Workbench Status=
 
When using the Workbench, you may want to change the status of some issues locally to your instance of Workbench. This will update the database tracking issues on your system; It does <b>not</b> affect what other users or system detect. This helps organize the issues locally quickly, for instance when externalizing strings with the Workbench.
 
   
  +
<br>
To change the status of an issue, select it, click the right mouse button, and click on the menu item. In the example below, the issue will be 'Ignored' locally.
 
   
  +
[[File:mark_prediction_as_false_bigger.png|800px]]
[[File:WorkbenchIgnore.gif]]
 
   
  +
<br>
<b>Note</b>: You can export a Globalyzer project with the result data. Any user or system can then import that project and it will set the result data status in the local database where the import happened.
 
   
  +
<b>Note</b>: After marking predictions, you can export a Prediction Report with the markings (or you can export your entire Globalyzer project with the result data). Any user or system can then import the Prediction Report (or import the entire Globalyzer project) and it will set the result data prediction in the local database where the import happened.
=Dashboard=
 
  +
The Dashboard shows the latest scan typically run on code in the repository. One can manage issues in the Dashboard. When logged in, a user can act on an issue: comment, assign, and set it to False Positive. The issue will not show up after the next update to the Dashboard.
 
  +
=Lingoport Dashboard=
  +
The Dashboard shows the latest scan typically run on code in the repository. One can manage issues in the Dashboard. When logged in, a user can act on an issue: comment, assign, or set it to False Positive. The issue will not show up after the next update to the Dashboard.
   
 
[[File:DashboardIgnore.gif]]
 
[[File:DashboardIgnore.gif]]
  +
   
 
That status is only in the Dashboard and does not affect issues found in the Workbench.
 
That status is only in the Dashboard and does not affect issues found in the Workbench.
   
 
False positives stick around as the file changes. Below are screenshots of a false positive issue which has persisted for 2 years (as of March 2016). The lines surrounding and including this issue have changed since it was first reported.
 
False positives stick around as the file changes. Below are screenshots of a false positive issue which has persisted for 2 years (as of March 2016). The lines surrounding and including this issue have changed since it was first reported.
  +
   
 
[[File:Dashboard_Line_below_issue_changed_later.png]]
 
[[File:Dashboard_Line_below_issue_changed_later.png]]
  +
  +
   
 
The issue above was 2 years old as of March 2016. The line just below it was changed a year after it was first detected.
 
The issue above was 2 years old as of March 2016. The line just below it was changed a year after it was first detected.
  +
   
 
[[File:Dashboard_old_issue_recent_update.png]]
 
[[File:Dashboard_old_issue_recent_update.png]]
  +
   
 
Additionally, the line itself was changed a few months later. The commit in question can be found [https://github.com/checkstyle/checkstyle/commit/d46134bc27a43ca9244e0c609f8ec044a43f36de here]. See lines 268 - 270.
 
Additionally, the line itself was changed a few months later. The commit in question can be found [https://github.com/checkstyle/checkstyle/commit/d46134bc27a43ca9244e0c609f8ec044a43f36de here]. See lines 268 - 270.
  +
  +
=False Positives Synchronization between Workbench and Dashboard=
  +
False positive issues can be identified in the Globalyzer Workbench or on the Lingoport Dashboard.
  +
This section discusses how to synchronize one with the other.
  +
  +
In the Workbench, you designate an issue as a False Positive by marking its prediction <code>False</code>.
  +
After marking issues <code>False</code>, you then export a Prediction Report. This saves the markings to a file that other Workbenches can import
  +
and that Lite can use when generating reports displayed on the Dashboard. The Dashboard knows to ignore issues with <code>False</code> predictions;
  +
those issues won't appear on the Dashboard.
  +
So false positive information moving from the Workbench to the Dashboard is fairly seamless, and uses the Prediction Report. No special explicit connection between the
  +
Workbench and Dashboard is necessary.
  +
  +
Going the other direction, from the Dashboard to the Workbench, does require an explicit connection between the Workbench and the Dashboard.
  +
On the Dashboard, you can set issues to False Positive and you can reopen issues that once were False Positive and make them active again.
  +
Reflecting these Dashboard changes back to the Workbench requires first setting up a connection between the Workbench and the Dashboard:
  +
  +
1.Connecting to Lingoport Dashboard
  +
  +
In the Workbench, click on the '''Window''' menu and choose '''Preferences''' from the dropdown menu. Select '''Globalyzer'''=>'''Lingoport Dashboard Settings'''.
  +
  +
<br>
  +
[[File:Login.png]]
  +
  +
  +
Specify the Server URL. Then, there are two ways to login. The default setting is use a Username/Password. If you want to login with a token, check "Use token to login". Click the Apply button to apply your settings and the Globalyzer Workbench will try to establish a connection to the Lingoport Dashboard with your credentials.
  +
  +
2. Set Dashboard Project Key for Workbench project
  +
  +
Click on the '''Project''' menu and choose '''Manage Globalyzer Projects''' from the dropdown menu. Select the Globalyzer Project you want to synchronize, then modify the '''Project Key at Dashboard''' value. Please note that you should enter the Project Key here instead of the Project Name; you can find the Project Key on the Dashboard.
  +
  +
<br>
  +
[[File:project_key.png]]
  +
<br>
  +
<br>
  +
  +
Once the connection is established, the Workbench can synchronize against the Dashboard by either selecting the <b>Project</b> menu or by right-clicking in <b>Scan Results</b> and then selecting <b>Reload Dashboard False Positives</b>.
  +
This action will identify all the False Positives on the Dashboard and update your Scan Results accordingly. False Positive issues from the Dashboard will be
  +
given <code>DFP</code> prediction values in the Workbench.
  +
  +
<br>
  +
[[File:Dfp_results.png]]
  +
<br>
  +
<br>
  +
  +
Confirmation information is written to the Workbench Console.
  +
  +
<br>
  +
[[File:Dfp_console.png]]
  +
<br>
   
 
=Feedback to Lingoport=
 
=Feedback to Lingoport=

Latest revision as of 23:41, 20 November 2019

What are False Positives?

Globalyzer scans the source code to detect candidate issues based on rules. Given some code, a rule may detect a candidate issue. In one context, that candidate issue may need to be corrected when in another context, the same issue may not need any work and is a false positive.

Let's look at a couple of lines of Java code which creates a date string based on a hard coded date format using SimpleDateFormat:

SimpleDateFormat formatter = new SimpleDateFormat("MM/dd/yy");
String dateString = formatter.format(new Date()); 

Using the default Java rule set, this code will raise i18n issues. If the dateString variable is user facing, this code is indeed an issue to be fixed (for more information, check your rule set help pages).

However, if the dateString variable is used for internal purposes, such as in a support log file, it should not be modified and is a false positive.

This section describes different ways to handle false positives.

Globalyzer Comment Tags

When you want to filter out a specific item in the code, one way is to add a Globalyzer comment on the offending line. This is for a specific item and only this one. If more code is written in the same way and Globalyzer should not detect the issue, a Rule Set modification may be better.

Let's look at a simple line of code where Globalyzer detects a candidate issue, here another embedded string:

   String marker = "Marker 15";


By adding the //$NON-NLS-L$ comment at the end of the line, Globalyzer will now ignore the issue:

   String marker = "Marker 15"; //$NON-NLS-L$ 


Since the code is pushed to the source control system, the next time a Workbench user or a script gets the latest code, Globalyzer will ignore the issue.

The default list of Globalyzer Tags is:

  • $NON-NLS-L$ : Ignores all issues on the line (all Embedded Strings, Locale-Sensitive Methods, General Patterns, and Static File References found on the line)
  • $NON-NLS-[number]$ : This is for Embedded Strings only and ignores one particular string on the line. For instance $NON-NLS-2$ ignores the second Embedded String on the line
  • GLOBALYZER_TODO : Indicates something to be done in i18n. It overrides/detects the line and sets a TODO status to it.
  • GLOBALYZER_IGNORE_NEXT_LINE: Ignores any issue on the line after this comment
  • GLOBALYZER_START_IGNORE : Ignores a block of issues until the GLOBALYZER_END_IGNORE tag comment
  • GLOBALYZER_END_IGNORE : Stops ignoring a block of issues, working with GLOBALYZER_START_IGNORE


Modifying Rules

Since Rules are what detect or filter out issues, the most common way to handle particular programming styles is to modify the rule set applied to scan a project's code.

Let's look at a string detected by Globalyzer:

SpecialTB.write("New user created");


Globalyzer identifies the string New user created as a candidate embedded string which should be moved out of the code and into a resource file. However, let's assume the developer knows that SpecialTB.write() is a method that writes some logging information into a database for potential debugging purposes. The Rule Set needs to be modified based on that knowledge so that this particular line and all other strings passed to this call will be filtered out, i.e. will not be detected as a candidate issue. In this instance, a String Method Filter would be created as shown below:


Add string method filter.png

Note: for the Java Rule Set only, we support Class or Variable Types, so you can filter strings when passed as a parameter to a particular object/method.

Any Globalyzer scan using this updated rule set will henceforth filter out the embedded string issue. Workbench instances and the nightly builds for instance would now filter out the issue.

You can also create and try new rules from the Globalyzer Workbench, and push the new rules to the server. To do so, right mouse click on an issue and select the Add Rule Sets Filters/Detection ... to show the dialog box.


Detailed Guide

Please visit the False Positive Filtering page for a detailed guide on creating false-positive filtering rules.

Globalyzer Workbench Prediction

Predictions are used to differentiate true issues from false positive ones. When using the Globalyzer Workbench, you may want to change the prediction of some issues locally to your instance of the Workbench. This will update the database tracking issues on your system; It does not affect what other users or system detect. This helps organize the issues locally quickly, for instance when externalizing strings with the Workbench. You can mark the prediction of issues as TRUE(T) if you know the issue is a true positive, FALSE(F) if you know the issue is a false positive, or PENDING(P) if you are unsure but you want to indicate that the issue needs further evaluation.

To change the prediction of an issue, select it, click the right mouse button, and click on the menu item. In the example below, the prediction of the issue will be 'F' locally.


Mark prediction as false bigger.png


Note: After marking predictions, you can export a Prediction Report with the markings (or you can export your entire Globalyzer project with the result data). Any user or system can then import the Prediction Report (or import the entire Globalyzer project) and it will set the result data prediction in the local database where the import happened.

Lingoport Dashboard

The Dashboard shows the latest scan typically run on code in the repository. One can manage issues in the Dashboard. When logged in, a user can act on an issue: comment, assign, or set it to False Positive. The issue will not show up after the next update to the Dashboard.

DashboardIgnore.gif


That status is only in the Dashboard and does not affect issues found in the Workbench.

False positives stick around as the file changes. Below are screenshots of a false positive issue which has persisted for 2 years (as of March 2016). The lines surrounding and including this issue have changed since it was first reported.


Dashboard Line below issue changed later.png


The issue above was 2 years old as of March 2016. The line just below it was changed a year after it was first detected.


Dashboard old issue recent update.png


Additionally, the line itself was changed a few months later. The commit in question can be found here. See lines 268 - 270.

False Positives Synchronization between Workbench and Dashboard

False positive issues can be identified in the Globalyzer Workbench or on the Lingoport Dashboard. This section discusses how to synchronize one with the other.

In the Workbench, you designate an issue as a False Positive by marking its prediction False. After marking issues False, you then export a Prediction Report. This saves the markings to a file that other Workbenches can import and that Lite can use when generating reports displayed on the Dashboard. The Dashboard knows to ignore issues with False predictions; those issues won't appear on the Dashboard. So false positive information moving from the Workbench to the Dashboard is fairly seamless, and uses the Prediction Report. No special explicit connection between the Workbench and Dashboard is necessary.

Going the other direction, from the Dashboard to the Workbench, does require an explicit connection between the Workbench and the Dashboard. On the Dashboard, you can set issues to False Positive and you can reopen issues that once were False Positive and make them active again. Reflecting these Dashboard changes back to the Workbench requires first setting up a connection between the Workbench and the Dashboard:

1.Connecting to Lingoport Dashboard

In the Workbench, click on the Window menu and choose Preferences from the dropdown menu. Select Globalyzer=>Lingoport Dashboard Settings.


Login.png


Specify the Server URL. Then, there are two ways to login. The default setting is use a Username/Password. If you want to login with a token, check "Use token to login". Click the Apply button to apply your settings and the Globalyzer Workbench will try to establish a connection to the Lingoport Dashboard with your credentials.

2. Set Dashboard Project Key for Workbench project

Click on the Project menu and choose Manage Globalyzer Projects from the dropdown menu. Select the Globalyzer Project you want to synchronize, then modify the Project Key at Dashboard value. Please note that you should enter the Project Key here instead of the Project Name; you can find the Project Key on the Dashboard.


Project key.png

Once the connection is established, the Workbench can synchronize against the Dashboard by either selecting the Project menu or by right-clicking in Scan Results and then selecting Reload Dashboard False Positives. This action will identify all the False Positives on the Dashboard and update your Scan Results accordingly. False Positive issues from the Dashboard will be given DFP prediction values in the Workbench.


Dfp results.png

Confirmation information is written to the Workbench Console.


Dfp console.png

Feedback to Lingoport

For issues you strongly believe are incorrectly detected in the default rule set, feel free to select those lines, press the right mouse button, and select "Report as False Positive". This will send an email with the lines of code and the issues that you feel should not be detected.

ReportAsFP.gif

This will help Lingoport refine the default rule set for future releases.