Difference between revisions of "Globalyzer Workbench"
(→Fixing i18n Issues FAQ) |
(→Globalyzer Workbench Project) |
||
(26 intermediate revisions by the same user not shown) | |||
Line 47: | Line 47: | ||
Yes. In fact this is quite common. One of the added values that our customers use Globalyzer for is as a repository for regular expression searches, which they can name, describe and share among their company’s developers. See [http://www.globalyzer.com/gzserver/help/usersguide/ruleSets.html#whatAreRuleSets Rule Sets] and scroll down to '''General Pattern Scan Rules''' for more details. |
Yes. In fact this is quite common. One of the added values that our customers use Globalyzer for is as a repository for regular expression searches, which they can name, describe and share among their company’s developers. See [http://www.globalyzer.com/gzserver/help/usersguide/ruleSets.html#whatAreRuleSets Rule Sets] and scroll down to '''General Pattern Scan Rules''' for more details. |
||
+ | |||
+ | === Can I restrict the Scanning Results to a Directory or to a File? === |
||
+ | Select directory or file in Project Explorer, right click and choose: |
||
+ | Globalyzer => View Scan Results For |
||
+ | Follow the above when you have a lot of results from scanning and you want to see only those issues relative to a sub-directory or even to one file. |
||
+ | |||
+ | === Showing the Rule Sets in a Browser or in the Workbench === |
||
+ | Window => Preferences => General => Web Browser |
||
+ | Choose '''Use internal web browser''' if you want to see the rule sets in the Workbench when you click '''View your Scan Rule Sets'''. |
||
+ | Choose '''Use external web browser''' if you want to see the rule sets in a separate Web browser. |
||
+ | |||
+ | To show the rulesets: Project -> View Globalyzer Rule Sets will log into the server and show the rulesetss. |
||
===Is there support for detecting UI alignment problems?=== |
===Is there support for detecting UI alignment problems?=== |
||
Line 89: | Line 101: | ||
Note: Dictionary filtering is turned on by default (see Window->Preferences->Globalyzer in the Globalyzer Workbench). <br> |
Note: Dictionary filtering is turned on by default (see Window->Preferences->Globalyzer in the Globalyzer Workbench). <br> |
||
− | Note: Globalyzer comes with its own dictionary. To add words to the dictionary, create a Company dictionary that is used in addition to the Globalyzer dictionary. See [[ |
+ | Note: Globalyzer comes with its own dictionary. To add words to the dictionary, create a Company dictionary that is used in addition to the Globalyzer dictionary. See [[Scanning_and_Filtering#How_can_I_add_words_to_the_dictionary_used_when_scanning.3F | How can I add words to the dictionary used when scanning?]] below for more information. |
all words found in dictionary (one or more words) |
all words found in dictionary (one or more words) |
||
Line 156: | Line 168: | ||
We assign as priority 2 or 3 issues which could be addressed later, such a sorting or font issues. These priorities are defaults which can be modified in the rule set. |
We assign as priority 2 or 3 issues which could be addressed later, such a sorting or font issues. These priorities are defaults which can be modified in the rule set. |
||
+ | === Concatenation Detection in Globalyzer Workbench === |
||
− | ===How should I filter false positive issues?=== |
||
+ | String Concatenation is marked with a <b>C</b> priority on detected strings according to the following rules: |
||
− | See [[False Positives | False Positives]] |
||
+ | * String literal either begins or ends with white space. |
||
− | Two of Globalyzer's main benefits are first in detecting possible internationalization (i18n) issues, and then in helping you hone down that list to only issues that will actually need code modifications to fix. There are three approaches that Globalyzer provides to address false positive issues in your Scan Results: |
||
+ | <pre> |
||
+ | String myString = " this"; |
||
+ | String myString2 = "that "; |
||
+ | </pre> |
||
+ | * String literal is preceded or followed by a concatenation operator. Concatenation operators vary by language. Any Locale Sensitive Method with priority 0 is marked as a concatenation issue. |
||
− | 1. Add filter rules to your Rule Set so that the next time you scan, Globalyzer will filter out those false positive detections. Since Rule Sets can be shared among team members and applied to any source code, this is a way to make a large scale effect with little effort and is our first recommendation for filtering Scan Results. |
||
+ | ** In Java, the concatenation operators are "+" and "+=". |
||
+ | <pre> |
||
+ | String myString3 = "hello" + " " |
||
+ | myString3 += "there"; |
||
+ | </pre> |
||
+ | * The String literal is a parameter to an Locale Sensitive Method with priority 0. |
||
− | 2. Change the Scan Results Status of false positive issues to be '''Invalid''' or '''Ignore'''. The Status settings are stored with the scanning results in Globalyzer's Client database on your local machine, so as long as you don't delete your scan results or modify the source code file significantly, the Status settings will be maintained during subsequent scans. We recommend this approach when there are specific results that you need to filter that don't lend themselves to Rule Set filters, and when developers prefer that you do not modify source code. |
||
+ | ** In Java, <code>concat</code> and <code>append</code> methods for <code>java.lang.String</code> are Locale Sensitive Methods with priority 0. |
||
+ | <pre> |
||
+ | myString3.append("something"); |
||
+ | myString3.concat("another thing"); |
||
+ | </pre> |
||
+ | When running a scan for that language, Globalyzer Workbench will detect the concatenated strings. Here is the Scan Results window with the Editor Pane pointing to one concatenation: |
||
− | 3. The last approach is to use one of the '''Ignore Comment''' options, which modifies the source code, and so is the most permanent resolution. Regardless whether a Rule Set filter is modified or Scan Results are deleted, or the source code file is significantly modified, Globalyzer will detect the ignore comment in the source code and skip over one or more lines of code that contain false positive i18n issues. |
||
+ | |||
+ | [[File:Concatenation In Workbench.png]] |
||
− | In addition, Globalyzer's Dictionary, enabled by default in the '''Window->Preferences->Globalyzer''' page, helps to filter and prioritize the Embedded Strings that are detected. For example, strings will be filtered if there are no matching Dictionary words. Conversely, depending on the number of matching versus non-matching words as well as other applied heuristics, Globalyzer will set the priority of each Scan Result, which aids in determining which strings will most likely be translatable strings. |
||
===Can you tell me more about inserting comments in my source code so that Globalyzer does not detect programmatic strings?=== |
===Can you tell me more about inserting comments in my source code so that Globalyzer does not detect programmatic strings?=== |
||
Line 270: | Line 297: | ||
To use your Company Dictionary for scanning, download the dictionary file from the Server and place in your Lingoport directory (C:\Lingoport on Windows, $HOME/lingoport on Linux, and /Applications/Lingoport on Mac). When scanning, Globalyzer will consider words in both the Lingoport Dictionary and your Company Dictionary. |
To use your Company Dictionary for scanning, download the dictionary file from the Server and place in your Lingoport directory (C:\Lingoport on Windows, $HOME/lingoport on Linux, and /Applications/Lingoport on Mac). When scanning, Globalyzer will consider words in both the Lingoport Dictionary and your Company Dictionary. |
||
− | ==Fixing i18n Issues |
+ | ==Fixing i18n Issues== |
===How do I get started on Fixing i18n Issues? === |
===How do I get started on Fixing i18n Issues? === |
||
Line 293: | Line 320: | ||
Static Files that might contain internationalization issues are usually handled by the original files and the localized files in locale-sensitive directories. The code is then refactored to retrieve the file based on the user's locale. Some examples of locale-sensitive static files are images with text, help files, video files, HTML files, and xml files that contain displayable text. |
Static Files that might contain internationalization issues are usually handled by the original files and the localized files in locale-sensitive directories. The code is then refactored to retrieve the file based on the user's locale. Some examples of locale-sensitive static files are images with text, help files, video files, HTML files, and xml files that contain displayable text. |
||
− | Globalyzer's Static File Reference Scan will scan the source code for references to these types of static files. After you've filtered out the false positives, you'll need to refactor the references so that they use the locale to determine which static file to load. |
+ | Globalyzer Workbench's Static File Reference Scan will scan the source code for references to these types of static files. After you've filtered out the false positives, you'll need to refactor the references so that they use the locale to determine which static file to load. |
+ | |||
+ | ===Will Globalyzer Workbench automatically externalize embedded strings?=== |
||
+ | |||
+ | Globalyzer Workbench has the capability of externalizing an unlimited number of embedded strings with just a few keystrokes. However, once you select the strings you want to externalize, you may view each reported issue in a snippet of the code and exclude it from the externalization process as needed. While batch externalization is a huge time-saver in your internationalization process, we do recommend checking the strings carefully first. |
||
+ | |||
+ | ==Globalyzer Workbench Project== |
||
+ | |||
+ | === Create a new Project=== |
||
+ | Note: A set of short tutorial videos is located on the [[About Globalyzer | About Globalyzer ]] page. One of the short videos focuses on creating a new project. |
||
+ | |||
+ | You can create a project by pointing to a source code directory in disk, or importing files in the Workbench workspace, which is more used by software engineers ready to refactor code. Importing files in the workspace can be done from files on disk or files in a source code control system, like Git or SVN. |
||
+ | ====Create a project by pointing to source code on disk==== |
||
+ | File => New => Project => General => Project |
||
+ | |||
+ | Uncheck '''Use Default Location''' and '''Browse''' to the location of your source code on disk. |
||
+ | Name the project and click '''Finish'''. If this is the first project, there are no Globalyzer configurations created yet to associate to this project. click '''New''' |
||
+ | To associate scanning logic to the new project click |
||
+ | Scan => Manage Scan |
||
+ | and follow the wizard. |
||
+ | |||
+ | ====Check out a project ==== |
||
+ | File => New => Project |
||
+ | and select the source code control system, such as Git or SVN if you have installed as an extension to the out of the box Workbench. |
||
+ | |||
+ | '''Note:''' To add an extension such as SVN, select: |
||
+ | Help => Install New Software |
||
+ | and, for SVN, add "SVN Subversive Updates - http://download.eclipse.org/technology/subversive/0.7/update-site" or another plugin or your choice. |
||
+ | |||
+ | |||
+ | === Spell Checking === |
||
+ | Window > Preferences => General => Editors => Text Editor => Spelling |
||
+ | |||
+ | will allow you to configure how to apply spell checking in a text editor. |
||
+ | |||
+ | ===Change the file association=== |
||
+ | Window => Preferences => General => Editors => File Associations |
||
+ | and associate the editor with the offending extension |
||
+ | |||
+ | |||
+ | ===What is the difference between a Workbench Project and a Globalyzer Project?=== |
||
+ | |||
+ | A '''Workbench Project''' is an Eclipse Project. A '''Globalyzer Project''' is a set of scans. Ideally you would have one Workbench Project associated with one Globalyzer Project. |
||
+ | |||
+ | ===How do I create a Workbench Project associated with a Globalyzer Project?=== |
||
+ | |||
+ | 1. Select '''File=>New''' to launch Eclipse's Project Wizard. |
||
+ | |||
+ | [[File: FileNewProject.jpg |600px]] |
||
+ | |||
+ | 2. Choose the type of Workbench Project you want to create. For example, if your application is a Java application, you can choose Java Project. If you don't have a plug-in for your application's programming language or you're planning to build and run your application outside the Globalyzer Workbench, select '''General=>Project''' and click '''Next'''. |
||
+ | |||
+ | [[File: GeneralProject.jpg |600px]] |
||
+ | |||
+ | 3. Enter a Project Name and source code Location in the New Project dialog. |
||
+ | |||
+ | [[File: CreateNewProject.jpg |600px]] |
||
+ | |||
+ | 4. Once the Eclipse Workbench Project is created, it will prompt you to create a Globalyzer Project and then Scans for the Project. |
||
+ | |||
+ | [[File: NewGlobalyzerProject.jpg |600px]] |
||
+ | [[File: ManageGlobalyzerScan.jpg |600px]] |
||
+ | |||
+ | ===I have a Globalyzer Project without a Workbench Project; how did this happen and how can I create a Workbench Project?=== |
||
+ | This can occur for a variety of reasons: |
||
+ | * You've imported a Globalyzer Project through the Globalyzer Command Line Client. |
||
+ | * You've deleted the associated Workbench Project. |
||
+ | |||
+ | Follow these steps to associate your Globalyzer Project with a new Workbench Project: |
||
+ | |||
+ | 1. Click on '''Project-->Manage Globalyzer Projects'''. This dialog lists all your Globalyzer Projects: Any Globalyzer Project that is associated with a Workbench Project will have a '''W''' icon next to its name. |
||
+ | |||
+ | 2. Click on the Globalyzer Project that you want to associate with a Workbench Project. |
||
+ | |||
+ | Note that once you've associated your Globalyzer Project with a Workbench Project, you'll no longer be able to change its name or source code location until you break the association. |
||
+ | |||
+ | 3. Click on the '''Associate''' button. This will bring up Eclipse's New Project wizard so that you can create the associated Workbench Project. Be sure to '''use the exact same Globalyzer Project name and source code location'''. Otherwise, the Eclipse Workbench Project won't be associated with the Globalyzer Project. Once you make the association, you'll see a '''W''' icon next to the Globalyzer Project's name. You'll also now be able to fully use Globalyzer features on the Project. |
||
+ | |||
+ | ===I have a Workbench Project without a Globalyzer Project, how did this happen and how can I create a Globalyzer Project?=== |
||
+ | This occurs when you've already set up an Eclipse Project for your application and now you want to use Globalyzer on it. |
||
+ | |||
+ | 1. Right-click on the Eclipse Workbench Project in the Navigation pane. |
||
+ | |||
+ | 2. Select '''Globalyzer=>Add/Remove Globalyzer Nature'''. This will create an associated Globalyzer Project and bring you into the New Scan Wizard. |
||
+ | |||
+ | [[File:Nature.png|600px]] |
||
+ | |||
+ | === What does "nature" mean? === |
||
+ | |||
+ | "Nature" is an Eclipse notion. Project natures act as tags on a project to indicate that a certain tool is used to operate on that project. When you create a project and add a '''Globalyzer Nature''' to the project, that project can have scans, etc. |
||
+ | |||
+ | ==What are False Positives?== |
||
+ | |||
+ | ===How should I filter false positive issues?=== |
||
+ | |||
+ | See [[False Positives | False Positives]] |
||
+ | |||
+ | Two of Globalyzer's main benefits are first in detecting possible internationalization (i18n) issues, and then in helping you hone down that list to only issues that will actually need code modifications to fix. There are three approaches that Globalyzer provides to address false positive issues in your Scan Results: |
||
+ | |||
+ | 1. Add filter rules to your Rule Set so that the next time you scan, Globalyzer will filter out those false positive detections. Since Rule Sets can be shared among team members and applied to any source code, this is a way to make a large scale effect with little effort and is our first recommendation for filtering Scan Results. |
||
+ | |||
+ | 2. Change the Scan Results Status of false positive issues to be '''Invalid''' or '''Ignore'''. The Status settings are stored with the scanning results in Globalyzer's Client database on your local machine, so as long as you don't delete your scan results or modify the source code file significantly, the Status settings will be maintained during subsequent scans. We recommend this approach when there are specific results that you need to filter that don't lend themselves to Rule Set filters, and when developers prefer that you do not modify source code. |
||
+ | |||
+ | 3. The last approach is to use one of the '''Ignore Comment''' options, which modifies the source code, and so is the most permanent resolution. Regardless whether a Rule Set filter is modified or Scan Results are deleted, or the source code file is significantly modified, Globalyzer will detect the ignore comment in the source code and skip over one or more lines of code that contain false positive i18n issues. |
||
+ | |||
+ | In addition, Globalyzer's Dictionary, enabled by default in the '''Window->Preferences->Globalyzer''' page, helps to filter and prioritize the Embedded Strings that are detected. For example, strings will be filtered if there are no matching Dictionary words. Conversely, depending on the number of matching versus non-matching words as well as other applied heuristics, Globalyzer will set the priority of each Scan Result, which aids in determining which strings will most likely be translatable strings. |
||
+ | |||
+ | Globalyzer scans the source code to detect candidate issues based on rules. Given some code, a rule may detect a candidate issue. In one context, that candidate issue may need to be corrected when in another context, the same issue may not need any work and is a false positive. |
||
+ | |||
+ | Let's look at a couple of lines of Java code which creates a date string based on a hard coded date format using '''<code>SimpleDateFormat</code>:''' |
||
+ | |||
+ | SimpleDateFormat formatter = new SimpleDateFormat("MM/dd/yy"); |
||
+ | String dateString = formatter.format(new Date()); |
||
+ | |||
+ | Using the default Java rule set, this code will raise [[Terms_and_Definitions#i18n|i18n]] issues. If the <code>dateString</code> variable is user facing, this code is indeed an issue to be fixed (for more information, check your rule set help pages). |
||
+ | |||
+ | However, if the <code>dateString</code> variable is used for internal purposes, such as in a support log file, it should not be modified and is a false positive. |
||
+ | |||
+ | This section describes different ways to handle false positives. |
||
+ | |||
+ | ===Modifying Rules=== |
||
+ | Since Rules are what detect or filter out issues, the most common way to handle particular programming styles is to modify the rule set applied to scan a project's code. |
||
+ | |||
+ | Let's look at a string detected by Globalyzer: |
||
+ | |||
+ | SpecialTB.write("New user created"); |
||
+ | |||
+ | Globalyzer identifies the string <code>New user created</code> as a candidate embedded string which should be moved out of the code and into a resource file. However, let's assume the developer knows that SpecialTB.write() is a method that writes some logging information into a database for potential debugging purposes. The Rule Set needs to be modified based on that knowledge so that this particular line and all other strings passed to this call will be filtered out, i.e. will not be detected as a candidate issue. In this instance, a '''String Method Filter''' would be created as shown below: |
||
+ | |||
+ | <br> |
||
+ | [[File:Add_string_method_filter.png]] |
||
+ | <br> |
||
+ | |||
+ | <b>Note:</b> for the Java Rule Set only, we support Class or Variable Types, so you can filter strings when |
||
+ | passed as a parameter to a particular object/method. |
||
+ | |||
+ | Any Globalyzer scan using this updated rule set will henceforth filter out the embedded string issue. Workbench instances and the nightly builds for instance would now filter out the issue. |
||
+ | |||
+ | You can also create and try new rules from the Globalyzer Workbench, and push the new rules to the server. To do so, right mouse click on an issue and select the '''Add Rule Sets Filters/Detection ...''' to show the dialog box. |
||
+ | |||
+ | === Detailed Guide === |
||
+ | |||
+ | Please visit the [[False_Positive_Filtering | False Positive Filtering]] page for a detailed guide on creating false-positive filtering rules. |
||
+ | |||
+ | ===Globalyzer Comment Tags=== |
||
+ | When you want to filter out a specific item in the code, one way is to add a Globalyzer comment on the offending line. This is for a specific item and only this one. If more code is written in the same way and Globalyzer should not detect the issue, a Rule Set modification may be better. |
||
+ | |||
+ | Let's look at a simple line of code where Globalyzer detects a candidate issue, here another embedded string: |
||
+ | |||
+ | String marker = "Marker 15"; |
||
+ | |||
+ | By adding the <b><code>//$NON-NLS-L$</code></b> comment at the end of the line, Globalyzer will now ignore the issue: |
||
+ | |||
+ | String marker = "Marker 15"; //$NON-NLS-L$ |
||
+ | |||
+ | Since the code is pushed to the source control system, the next time a Workbench user or a script gets the latest code, Globalyzer will ignore the issue. |
||
+ | |||
+ | The default list of Globalyzer Tags is: |
||
+ | * <code>$NON-NLS-L$</code> : Ignores any issue on the line |
||
+ | * <code>$NON-NLS-[number]$</code> : Ignores one particular issue on the line. For instance $NON-NLS-2$ ignores the second issue on the line |
||
+ | * <code>GLOBALYZER_TODO</code> : Indicates something to be done in i18n. It overrides/detects the line and sets a TODO status to it. |
||
+ | * <code>GLOBALYZER_IGNORE_NEXT_LINE</code>: Ignores any issue on the line after this comment |
||
+ | * <code>GLOBALYZER_START_IGNORE</code> : Ignores a block of issues until the GLOBALYZER_END_IGNORE tag comment |
||
+ | * <code>GLOBALYZER_END_IGNORE</code> : Stops ignoring a block of issues, working with GLOBALYZER_START_IGNORE |
||
+ | |||
+ | ===Globalyzer Workbench Prediction=== |
||
+ | Predictions are used to differentiate true issues from false positive ones. |
||
+ | When using the Globalyzer Workbench, you may want to change the prediction of some issues locally to your instance of the Workbench. |
||
+ | This will update the database tracking issues on your system; It does <b>not</b> affect what other users or system detect. |
||
+ | This helps organize the issues locally quickly, for instance when externalizing strings with the Workbench. You can mark the prediction of issues as <code>TRUE(T)</code> if you |
||
+ | know the issue is a true positive, <code>FALSE(F)</code> if you know the issue is a false positive, or <code>PENDING(P)</code> if you are unsure but you want to indicate |
||
+ | that the issue needs further evaluation. |
||
+ | |||
+ | To change the prediction of an issue, select it, click the right mouse button, and click on the menu item. In the example below, the prediction of the issue will be 'F' locally. |
||
+ | |||
+ | <br> |
||
+ | |||
+ | [[File:mark_prediction_as_false_bigger.png|800px]] |
||
+ | |||
+ | <br> |
||
+ | |||
+ | <b>Note</b>: After marking predictions, you can export a Prediction Report with the markings (or you can export your entire Globalyzer project with the result data). Any user or system can then import the Prediction Report (or import the entire Globalyzer project) and it will set the result data prediction in the local database where the import happened. |
||
+ | |||
+ | In the Workbench, you designate an issue as a False Positive by marking its prediction <code>False</code>. |
||
+ | After marking issues <code>False</code>, you then export a Prediction Report. This saves the markings to a file that other Workbenches can import |
||
+ | and that Lite can use when generating reports displayed on the Dashboard. The Dashboard knows to ignore issues with <code>False</code> predictions; |
||
+ | those issues won't appear on the Dashboard. |
||
+ | So false positive information moving from the Workbench to the Dashboard is fairly seamless, and uses the Prediction Report. No special explicit connection between the |
||
+ | Workbench and Dashboard is necessary. |
||
+ | |||
+ | Going the other direction, from the Dashboard to the Workbench, does require an explicit connection between the Workbench and the Dashboard. |
||
+ | On the Dashboard, you can set issues to False Positive and you can reopen issues that once were False Positive and make them active again. |
||
+ | Reflecting these Dashboard changes back to the Workbench requires first setting up a connection between the Workbench and the Dashboard: |
||
+ | |||
+ | 1.Connecting to Lingoport Dashboard |
||
+ | |||
+ | In the Workbench, click on the '''Window''' menu and choose '''Preferences''' from the dropdown menu. Select '''Globalyzer'''=>'''Lingoport Dashboard Settings'''. |
||
+ | |||
+ | <br> |
||
+ | [[File:Login.png]] |
||
+ | |||
+ | |||
+ | Specify the Server URL. Then, there are two ways to login. The default setting is use a Username/Password. If you want to login with a token, check "Use token to login". Click the Apply button to apply your settings and the Globalyzer Workbench will try to establish a connection to the Lingoport Dashboard with your credentials. |
||
+ | |||
+ | 2. Set Dashboard Project Key for Workbench project |
||
+ | |||
+ | Click on the '''Project''' menu and choose '''Manage Globalyzer Projects''' from the dropdown menu. Select the Globalyzer Project you want to synchronize, then modify the '''Project Key at Dashboard''' value. Please note that you should enter the Project Key here instead of the Project Name; you can find the Project Key on the Dashboard. |
||
+ | |||
+ | <br> |
||
+ | [[File:project_key.png]] |
||
+ | <br> |
||
+ | <br> |
||
+ | |||
+ | Once the connection is established, the Workbench can synchronize against the Dashboard by either selecting the <b>Project</b> menu or by right-clicking in <b>Scan Results</b> and then selecting <b>Reload Dashboard False Positives</b>. |
||
+ | This action will identify all the False Positives on the Dashboard and update your Scan Results accordingly. False Positive issues from the Dashboard will be |
||
+ | given <code>DFP</code> prediction values in the Workbench. |
||
+ | |||
+ | <br> |
||
+ | [[File:Dfp_results.png]] |
||
+ | <br> |
||
+ | <br> |
||
+ | |||
+ | Confirmation information is written to the Workbench Console. |
||
+ | |||
+ | <br> |
||
+ | [[File:Dfp_console.png]] |
||
+ | <br> |
||
+ | |||
+ | ===Feedback to Lingoport=== |
||
+ | For issues you strongly believe are incorrectly detected in the default rule set, feel free to select those lines, press the right mouse button, and select "Report as False Positive". This will send an email with the lines of code and the issues that you feel should not be detected. |
||
+ | |||
+ | [[File:ReportAsFP.gif]] |
||
+ | |||
+ | This will help Lingoport refine the default rule set for future releases. |
||
+ | ==Machine Learning== |
||
+ | Machine Learning prediction is a [[Terms_and_Definitions#GlobalyzerWorkbench|Globalyzer Workbench]] and [[Terms_and_Definitions#GlobalyzerLite|Globalyzer Lite]] feature that helps users more quickly identify the real issues in their source code. We suggest applying [[Terms_and_Definitions#MachineLearning|Machine Learning]] as a follow-up step to scanning with Rule Sets. It helps to determine which candidate issues are indeed [[Terms_and_Definitions#i18n|i18n]] issues. |
||
− | ===Will Globalyzer automatically externalize embedded strings?=== |
||
+ | Please see the [https://wiki.lingoport.com/Machine_Learning Machine Learning] page for detailed instructions on how to use Lingoport's Machine Learning implementation. |
||
− | Globalyzer has the capability of externalizing an unlimited number of embedded strings with just a few keystrokes. However, once you select the strings you want to externalize, you may view each reported issue in a snippet of the code and exclude it from the externalization process as needed. While batch externalization is a huge time-saver in your internationalization process, we do recommend checking the strings carefully first. |
Latest revision as of 21:12, 14 June 2019
Contents
- 1 General
- 1.1 Can Globalyzer Workbench help me build resource files and externalize strings from my source code?
- 1.2 How do I install Globalyzer Workbench ?
- 1.3 Is Globalyzer Workbench automatically (automagically) fixing my code? Is Globalyzer Workbench adding anything proprietary?
- 1.4 What does "nature" mean?
- 1.5 In Globalyzer Workbench, after the string extraction, can a developer tell what string is associated with the string key other than opening up the resource file?
- 1.6 Can a developer customize the result list – for example, have the freedom to move entries to to-do list or ignore list, etc.?
- 2 Scanning and Filtering
- 2.1 Can I use Globalyzer to look for issues in my code that have nothing to do with internationalization?
- 2.2 Can I restrict the Scanning Results to a Directory or to a File?
- 2.3 Showing the Rule Sets in a Browser or in the Workbench
- 2.4 Is there support for detecting UI alignment problems?
- 2.5 What is the difference between the “Ignore” and “Invalid” Status?
- 2.6 How does Globalyzer determine whether or not an embedded string is an internationalization issue?
- 2.7 How are i18n issue priorities set?
- 2.8 Concatenation Detection in Globalyzer Workbench
- 2.9 Can you tell me more about inserting comments in my source code so that Globalyzer does not detect programmatic strings?
- 2.10 Could you provide some sense of scanning performance?
- 2.11 How can I scan code written in programming languages Globalyzer does not support out of the box?
- 2.12 How can I add words to the dictionary used when scanning?
- 3 Fixing i18n Issues
- 3.1 How do I get started on Fixing i18n Issues?
- 3.2 How can I tell if I am using the correct method call for a supported locale?
- 3.3 Will Globalyzer help me adapt my application to support Unicode?
- 3.4 How should I fix locale-sensitive Static File references?
- 3.5 Will Globalyzer Workbench automatically externalize embedded strings?
- 4 Globalyzer Workbench Project
- 4.1 Create a new Project
- 4.2 Spell Checking
- 4.3 Change the file association
- 4.4 What is the difference between a Workbench Project and a Globalyzer Project?
- 4.5 How do I create a Workbench Project associated with a Globalyzer Project?
- 4.6 I have a Globalyzer Project without a Workbench Project; how did this happen and how can I create a Workbench Project?
- 4.7 I have a Workbench Project without a Globalyzer Project, how did this happen and how can I create a Globalyzer Project?
- 4.8 What does "nature" mean?
- 5 What are False Positives?
- 6 Machine Learning
General
Can Globalyzer Workbench help me build resource files and externalize strings from my source code?
Yes it can, and it will add great productivity to that long and tedious task. It is common for developers to be able to comfortably externalize 1,000 strings and more per day using Globalyzer Workbench. Globalyzer Workbench lets you see the string within your code and manages the whole externalization process. It is customizable, giving the developer complete control while aiding productivity.
How do I install Globalyzer Workbench ?
To install Globalyzer Workbench, you first need a Globalyzer account (see this FAQ). Once you have an account, you log into Globalyzer Server and at the bottom of your home page there is a link to download Globalyzer Workbench.
Globalyzer Workbench is a standalone Eclipse Rich Client Platform (RCP) application. In addition to many internationalization features, the Workbench benefits from Eclipse's support for integrating with source control, programming language plug-ins and any other plug-in to assist with your development.
Please see installation for complete Globalyzer Workbench download/installation instructions.
Is Globalyzer Workbench automatically (automagically) fixing my code? Is Globalyzer Workbench adding anything proprietary?
No, Globalyzer Workbench does not make changes without your knowledge and does not add any proprietary elements to your source code. However, once you do approve the fix for one or more internationalization issues, Globalyzer Workbench does have highly productive capabilities to speed up repetitive internationalization tasks, such as string externalization.
What does "nature" mean?
"Nature" is an Eclipse notion. Project natures act as tags on a project to indicate that a certain tool is used to operate on that project. When you create a project and add a Globalyzer Nature to the project, that project can have scans, etc.
In Globalyzer Workbench, after the string extraction, can a developer tell what string is associated with the string key other than opening up the resource file?
If you select an externalized string key in the source code, you can right click and select Globalyzer->Resources->Go to Resource Definition
It will bring up the resource file with that entry highlighted.
Also, if you set the preference to Window -> Preferences -> Globalyzer - Automatically Insert Externalized String Comment, when strings are externalized, a comment will be added to the source file with the text of the string that was externalized.
Can a developer customize the result list – for example, have the freedom to move entries to to-do list or ignore list, etc.?
Yes, scan result statuses can be changed. Select one or more scan results, right-click and select the new status.
Scanning and Filtering
Can I use Globalyzer to look for issues in my code that have nothing to do with internationalization?
Yes. In fact this is quite common. One of the added values that our customers use Globalyzer for is as a repository for regular expression searches, which they can name, describe and share among their company’s developers. See Rule Sets and scroll down to General Pattern Scan Rules for more details.
Can I restrict the Scanning Results to a Directory or to a File?
Select directory or file in Project Explorer, right click and choose:
Globalyzer => View Scan Results For
Follow the above when you have a lot of results from scanning and you want to see only those issues relative to a sub-directory or even to one file.
Showing the Rule Sets in a Browser or in the Workbench
Window => Preferences => General => Web Browser
Choose Use internal web browser if you want to see the rule sets in the Workbench when you click View your Scan Rule Sets. Choose Use external web browser if you want to see the rule sets in a separate Web browser.
To show the rulesets: Project -> View Globalyzer Rule Sets will log into the server and show the rulesetss.
Is there support for detecting UI alignment problems?
This would be handled by adding new General Pattern Rules to your Rule Set that detect fixed widths.
What is the difference between the “Ignore” and “Invalid” Status?
These status choices have been deprecated, but can still be used, although there are different ways to do things.
These status choices can have any meaning, and are just a way to further break up the scan results.
That said, we think that the Invalid status should be used in the cases where the scan detection is not correct. For example, say Globalyzer’s Embedded String scan found a detection that wasn’t a UI embedded string (for example, a programmatic string). In this case, you would set the result to the Invalid status.
The Ignore status, on the other hand, would be used for embedded strings that do appear in the UI but that you don’t translate, for example, file extensions or program names.
The TODO is another status that you could apply to scan results that you want to deal with at a later time. In the case of the Locale-Sensitive Method scan, you could apply the Ignore status to results where the method usage is not an issue and use the TODO status if you want to look at the method later.
How does Globalyzer determine whether or not an embedded string is an internationalization issue?
During the Embedded String scanning process, Globalyzer will detect all strings in the source code. Then, the Globalyzer Embedded String Filtering Rules are used to eliminate the programmatic strings, or strings that would not require translation. So, the scanning work is primarily in filtering out strings, rather than in detecting them.
How are i18n issue priorities set?
Each active issue has a priority. The priority ranges between 1 and 5, 1 being the highest - i.e. the most likely to be an internationalization issue. In addition, Strings can be marked as 'Concatenated', or priority 0. For Locale-Sensitive Methods, Static File References, and General Patterns, the priority is configured in the detection rules of the rule set. For Embedded Strings, priorities can be configured in retention rules, such as String Content Patterns, String Method Patterns, and String Variable Patterns.
If a string is detected by these rules, the configured priority is assigned to the issue. If not, Globalyzer assigns a priority based on the content of the string. For example, a string containing words found in the dictionary will get a higher priority (more likely to be displayed to the end user and therefore more likely to require externalization) than one with words not found in the dictionary. Other considerations for assigning priorities to Embedded Strings include capitalization and punctuation found in the string. Priorities help focus efforts on the most important active issues.
Globalyzer Workbench also allows the user to assign different priorities to issues.
String Priorities
For strings, here is the algorithm we use to determine priority:
Priority: 0 / Concatenation
Any string which is not filtered and is concatenated is marked as 0/Concatenated.
Priority 1:
- if dictionary filtering is ON:
Note: Dictionary filtering is turned on by default (see Window->Preferences->Globalyzer in the Globalyzer Workbench).
Note: Globalyzer comes with its own dictionary. To add words to the dictionary, create a Company dictionary that is used in addition to the Globalyzer dictionary. See How can I add words to the dictionary used when scanning? below for more information.
all words found in dictionary (one or more words) and first word character is uppercase (and second character of first word is not uppercase) and string doesn't contain non word characters (that is, string only contains letters, numbers, and/or white space)
- if dictionary filter is OFF:
first word character is upper case (and second character of first word is lowercase) and string doesn't contain non word characters
Priority 2 (not Priority 1 and):
- if dictionary filtering is ON:
all words found in dictionary and first word character is lowercase (i.e. first letter is lowercase) and number of words is 2 or more and string doesn't contain non word characters
OR
all words found in dictionary and first word character is uppercase (and second character of first word is lowercase) and string contains non word characters
- if dictionary filter is OFF:
first word character is lowercase (i.e. first letter is lowercase) and number of words is >= 2 and string doesn't contain non word characters
OR
first word character is uppercase (and second character of first word is lowercase) and string contains non word characters
Priority 3 (not Priority 1, 2 and):
- if dictionary filtering is ON:
all words found in dictionary (including one word phrase) and first word character is lower case (i.e. first letter is lowercase) and string contains non word characters
- if dictionary filtering is OFF:
first word character is lower case (i.e. first letter is lowercase) and string contains non word characters
Priority 4 (not Priority 1, 2, 3 and):
- if dictionary filter is ON:
at least one word found in dictionary and number of words is 2 or more
Priority 5: not Priority 1, 2, 3, 4
Locale-Sensitive Methods Priorities
For Locale-Sensitive Methods, General Patterns, and Static File References, our general approach is to assign it a priority 1 if the application could break if the i18n issue isn't addressed. For example, an encoding issue where you would see character corruption.
We also assign priority 1 to methods that are likely to have an i18n issue, such as date/time/number formatting methods.
We assign as priority 2 or 3 issues which could be addressed later, such a sorting or font issues. These priorities are defaults which can be modified in the rule set.
Concatenation Detection in Globalyzer Workbench
String Concatenation is marked with a C priority on detected strings according to the following rules:
- String literal either begins or ends with white space.
String myString = " this"; String myString2 = "that ";
- String literal is preceded or followed by a concatenation operator. Concatenation operators vary by language. Any Locale Sensitive Method with priority 0 is marked as a concatenation issue.
- In Java, the concatenation operators are "+" and "+=".
String myString3 = "hello" + " " myString3 += "there";
- The String literal is a parameter to an Locale Sensitive Method with priority 0.
- In Java,
concat
andappend
methods forjava.lang.String
are Locale Sensitive Methods with priority 0.
- In Java,
myString3.append("something"); myString3.concat("another thing");
When running a scan for that language, Globalyzer Workbench will detect the concatenated strings. Here is the Scan Results window with the Editor Pane pointing to one concatenation:
Can you tell me more about inserting comments in my source code so that Globalyzer does not detect programmatic strings?
The following comments can be used to tell Globalyzer some issues are not to be detected:
1. Use Eclipse style comments: select the rows that you want to ignore in the Scan Results table; Then right-click and choose Ignore this Issue. Globalyzer will insert comments of the form $NON-NLS-[n]$ at the end of the line, where "n" refers to the nth issue on the line. For example, if there were 2 embedded strings on the line that you wanted to ignore, Globalyzer Workbench would insert: $NON-NLS-1$ $NON-NLS-2$ for the two times that you chose Ignore this Issue.
4. Ignore all the issues on a line, choose Ignore this Line. Globalyzer Workbench will insert comments with $NON-NLS-L$ comments at the end of the line. Subsequent scans will ignore the lines with $NON-NLS-L$ comments.
1. Insert a comment in the source code that indicates the next line does not require scanning. The comment itself is customizable in Globalyzer's project properties setting. By default it is GLOBALYZER_IGNORE_NEXT_LINE.
2. Skip scanning a block of code by using a comment to start and a comment to end the "skipped block". These comment commands are also customizable. By default, the keywords are: GLOBALYZER_START_IGNORE and GLOBALYZER_END_IGNORE.
Note: Globalyzer Workbench also has a way to ignore issues without modifying the source – using the Status setting in the Scan Results view. A Status on the Workbench is local to that Workbench instance. It is not seen by Lite for instance. To move the Status to the Dashboard, use the Dashboard False Positive Workbench function (See False Positives ). As long as the files don’t change too much, the Status will keep track of the issues. If you want to share your Status settings with other team members, you may choose to
- export your Globalyzer Project (with Scan Results checked): they will need to import the project. You will also need to make sure that the Globalyzer Server Rule Sets used by the project are set to shared so that they're available to the entire team.
- Use Dashboard False Positives to centralize all False Positives on the Dashboard. See the False Positives Synchronization section in the False Positives page.
Could you provide some sense of scanning performance?
Performance will depend on the machine running the scans. Lingoport performance tests are geared towards a Continuous Globalization process where a Jenkins type system executes scanning jobs. The following gives a performance benchmark for a specific machines on the two major operations when running in a Continuous Globalyzer:
- Login in
- Running the scans, with or without rule sets reload
Benchmark System Info: Operating System Name: Windows 8.1 Operating System Type: amd64 Operating System Version: 6.3 Processor Identifier: Intel64 Family 6 Model 58 Stepping 9, GenuineIntel Number of Processors: 4 Available processors (cores): 4 Free memory (bytes): 79497416 Maximum memory (bytes): 1407713280 Total memory (bytes): 94896128 Number of Rule Sets: 10 Login Benchmark: 559 milliseconds Login Benchmark: 324 milliseconds Login Benchmark: 336 milliseconds Login Benchmark: 313 milliseconds Login Benchmark: 307 milliseconds Login Benchmark: 306 milliseconds Login Benchmark: 313 milliseconds Login Benchmark: 300 milliseconds Login Benchmark: 303 milliseconds Login Benchmark: 290 milliseconds Execute Java Scan Benchmark (927 lines of code, 9 files, 190 active results): Reloaded rule sets Executing java scan: 2073 milliseconds Executing java scan: 746 milliseconds Executing java scan: 557 milliseconds Executing java scan: 553 milliseconds Executing java scan: 533 milliseconds Executing java scan: 541 milliseconds Executing java scan: 520 milliseconds Executing java scan: 513 milliseconds Executing java scan: 524 milliseconds Executing java scan: 516 milliseconds Reloaded rule sets Executing java scan: 652 milliseconds Executing java scan: 574 milliseconds Executing java scan: 526 milliseconds Executing java scan: 493 milliseconds Executing java scan: 488 milliseconds Executing java scan: 499 milliseconds Executing java scan: 486 milliseconds Executing java scan: 481 milliseconds Executing java scan: 475 milliseconds Executing java scan: 470 milliseconds Reloaded rule sets Executing java scan: 611 milliseconds Executing java scan: 478 milliseconds Executing java scan: 465 milliseconds Executing java scan: 472 milliseconds Executing java scan: 464 milliseconds Executing java scan: 483 milliseconds Executing java scan: 469 milliseconds Executing java scan: 455 milliseconds Executing java scan: 459 milliseconds Executing java scan: 459 milliseconds
How can I scan code written in programming languages Globalyzer does not support out of the box?
Globalyzer provides a set of internationalization rules in support of the main programming languages. In case your code is written in a language that is not part of the out of the box rules, you can do the following:
- Call Lingoport to include a new set of rules for you: This may or may not be a consulting effort by Lingoport
- Find a supported programming language whose constructs are close to your own and create your own rule set. To do so, please visit the Scanning Unsupported Languages page.
How can I add words to the dictionary used when scanning?
If the preference Filter Strings without Dictionary Words is checked (see Window->Preferences->Globalyzer), Globalyzer will automatically filter strings that contain no dictionary words. That is, the string itself is parsed into words, and if none of the words are found in the dictionary, the string is filtered. If at least one of the words is found in the dictionary, the string is potentially active.
The dictionary is maintained by Lingoport. If there are words you would like added to the dictionary, you can create a Company Dictionary on the Globalyzer Server and add words to that. There is one Company Dictionary per company, so all users in the company will see the same dictionary.
To add words to the Company Dictionary, download the dictionary file from the Globalyzer Server, modify the file locally (one word per line), and then upload the dictionary file to the Server.
To use your Company Dictionary for scanning, download the dictionary file from the Server and place in your Lingoport directory (C:\Lingoport on Windows, $HOME/lingoport on Linux, and /Applications/Lingoport on Mac). When scanning, Globalyzer will consider words in both the Lingoport Dictionary and your Company Dictionary.
Fixing i18n Issues
How do I get started on Fixing i18n Issues?
A set of short tutorial videos is available on the About Globalyzer page. These were made several years ago, but the process is still the same.
How can I tell if I am using the correct method call for a supported locale?
For many of its supported languages, Globalyzer will provide you with information and direction for refactoring methods and functions, giving examples and cross references when appropriate. It’s probably the largest on-line internationalization help system existing anywhere.
Will Globalyzer help me adapt my application to support Unicode?
Yes, as is necessary to your particular programming language. You will have a number of character encoding support choices.
How should I fix locale-sensitive Static File references?
Static Files that might contain internationalization issues are usually handled by the original files and the localized files in locale-sensitive directories. The code is then refactored to retrieve the file based on the user's locale. Some examples of locale-sensitive static files are images with text, help files, video files, HTML files, and xml files that contain displayable text.
Globalyzer Workbench's Static File Reference Scan will scan the source code for references to these types of static files. After you've filtered out the false positives, you'll need to refactor the references so that they use the locale to determine which static file to load.
Will Globalyzer Workbench automatically externalize embedded strings?
Globalyzer Workbench has the capability of externalizing an unlimited number of embedded strings with just a few keystrokes. However, once you select the strings you want to externalize, you may view each reported issue in a snippet of the code and exclude it from the externalization process as needed. While batch externalization is a huge time-saver in your internationalization process, we do recommend checking the strings carefully first.
Globalyzer Workbench Project
Create a new Project
Note: A set of short tutorial videos is located on the About Globalyzer page. One of the short videos focuses on creating a new project.
You can create a project by pointing to a source code directory in disk, or importing files in the Workbench workspace, which is more used by software engineers ready to refactor code. Importing files in the workspace can be done from files on disk or files in a source code control system, like Git or SVN.
Create a project by pointing to source code on disk
File => New => Project => General => Project
Uncheck Use Default Location and Browse to the location of your source code on disk. Name the project and click Finish. If this is the first project, there are no Globalyzer configurations created yet to associate to this project. click New To associate scanning logic to the new project click
Scan => Manage Scan
and follow the wizard.
Check out a project
File => New => Project
and select the source code control system, such as Git or SVN if you have installed as an extension to the out of the box Workbench.
Note: To add an extension such as SVN, select:
Help => Install New Software
and, for SVN, add "SVN Subversive Updates - http://download.eclipse.org/technology/subversive/0.7/update-site" or another plugin or your choice.
Spell Checking
Window > Preferences => General => Editors => Text Editor => Spelling
will allow you to configure how to apply spell checking in a text editor.
Change the file association
Window => Preferences => General => Editors => File Associations
and associate the editor with the offending extension
What is the difference between a Workbench Project and a Globalyzer Project?
A Workbench Project is an Eclipse Project. A Globalyzer Project is a set of scans. Ideally you would have one Workbench Project associated with one Globalyzer Project.
How do I create a Workbench Project associated with a Globalyzer Project?
1. Select File=>New to launch Eclipse's Project Wizard.
2. Choose the type of Workbench Project you want to create. For example, if your application is a Java application, you can choose Java Project. If you don't have a plug-in for your application's programming language or you're planning to build and run your application outside the Globalyzer Workbench, select General=>Project and click Next.
3. Enter a Project Name and source code Location in the New Project dialog.
4. Once the Eclipse Workbench Project is created, it will prompt you to create a Globalyzer Project and then Scans for the Project.
I have a Globalyzer Project without a Workbench Project; how did this happen and how can I create a Workbench Project?
This can occur for a variety of reasons:
- You've imported a Globalyzer Project through the Globalyzer Command Line Client.
- You've deleted the associated Workbench Project.
Follow these steps to associate your Globalyzer Project with a new Workbench Project:
1. Click on Project-->Manage Globalyzer Projects. This dialog lists all your Globalyzer Projects: Any Globalyzer Project that is associated with a Workbench Project will have a W icon next to its name.
2. Click on the Globalyzer Project that you want to associate with a Workbench Project.
Note that once you've associated your Globalyzer Project with a Workbench Project, you'll no longer be able to change its name or source code location until you break the association.
3. Click on the Associate button. This will bring up Eclipse's New Project wizard so that you can create the associated Workbench Project. Be sure to use the exact same Globalyzer Project name and source code location. Otherwise, the Eclipse Workbench Project won't be associated with the Globalyzer Project. Once you make the association, you'll see a W icon next to the Globalyzer Project's name. You'll also now be able to fully use Globalyzer features on the Project.
I have a Workbench Project without a Globalyzer Project, how did this happen and how can I create a Globalyzer Project?
This occurs when you've already set up an Eclipse Project for your application and now you want to use Globalyzer on it.
1. Right-click on the Eclipse Workbench Project in the Navigation pane.
2. Select Globalyzer=>Add/Remove Globalyzer Nature. This will create an associated Globalyzer Project and bring you into the New Scan Wizard.
What does "nature" mean?
"Nature" is an Eclipse notion. Project natures act as tags on a project to indicate that a certain tool is used to operate on that project. When you create a project and add a Globalyzer Nature to the project, that project can have scans, etc.
What are False Positives?
How should I filter false positive issues?
See False Positives
Two of Globalyzer's main benefits are first in detecting possible internationalization (i18n) issues, and then in helping you hone down that list to only issues that will actually need code modifications to fix. There are three approaches that Globalyzer provides to address false positive issues in your Scan Results:
1. Add filter rules to your Rule Set so that the next time you scan, Globalyzer will filter out those false positive detections. Since Rule Sets can be shared among team members and applied to any source code, this is a way to make a large scale effect with little effort and is our first recommendation for filtering Scan Results.
2. Change the Scan Results Status of false positive issues to be Invalid or Ignore. The Status settings are stored with the scanning results in Globalyzer's Client database on your local machine, so as long as you don't delete your scan results or modify the source code file significantly, the Status settings will be maintained during subsequent scans. We recommend this approach when there are specific results that you need to filter that don't lend themselves to Rule Set filters, and when developers prefer that you do not modify source code.
3. The last approach is to use one of the Ignore Comment options, which modifies the source code, and so is the most permanent resolution. Regardless whether a Rule Set filter is modified or Scan Results are deleted, or the source code file is significantly modified, Globalyzer will detect the ignore comment in the source code and skip over one or more lines of code that contain false positive i18n issues.
In addition, Globalyzer's Dictionary, enabled by default in the Window->Preferences->Globalyzer page, helps to filter and prioritize the Embedded Strings that are detected. For example, strings will be filtered if there are no matching Dictionary words. Conversely, depending on the number of matching versus non-matching words as well as other applied heuristics, Globalyzer will set the priority of each Scan Result, which aids in determining which strings will most likely be translatable strings.
Globalyzer scans the source code to detect candidate issues based on rules. Given some code, a rule may detect a candidate issue. In one context, that candidate issue may need to be corrected when in another context, the same issue may not need any work and is a false positive.
Let's look at a couple of lines of Java code which creates a date string based on a hard coded date format using SimpleDateFormat
:
SimpleDateFormat formatter = new SimpleDateFormat("MM/dd/yy"); String dateString = formatter.format(new Date());
Using the default Java rule set, this code will raise i18n issues. If the dateString
variable is user facing, this code is indeed an issue to be fixed (for more information, check your rule set help pages).
However, if the dateString
variable is used for internal purposes, such as in a support log file, it should not be modified and is a false positive.
This section describes different ways to handle false positives.
Modifying Rules
Since Rules are what detect or filter out issues, the most common way to handle particular programming styles is to modify the rule set applied to scan a project's code.
Let's look at a string detected by Globalyzer:
SpecialTB.write("New user created");
Globalyzer identifies the string New user created
as a candidate embedded string which should be moved out of the code and into a resource file. However, let's assume the developer knows that SpecialTB.write() is a method that writes some logging information into a database for potential debugging purposes. The Rule Set needs to be modified based on that knowledge so that this particular line and all other strings passed to this call will be filtered out, i.e. will not be detected as a candidate issue. In this instance, a String Method Filter would be created as shown below:
Note: for the Java Rule Set only, we support Class or Variable Types, so you can filter strings when passed as a parameter to a particular object/method.
Any Globalyzer scan using this updated rule set will henceforth filter out the embedded string issue. Workbench instances and the nightly builds for instance would now filter out the issue.
You can also create and try new rules from the Globalyzer Workbench, and push the new rules to the server. To do so, right mouse click on an issue and select the Add Rule Sets Filters/Detection ... to show the dialog box.
Detailed Guide
Please visit the False Positive Filtering page for a detailed guide on creating false-positive filtering rules.
Globalyzer Comment Tags
When you want to filter out a specific item in the code, one way is to add a Globalyzer comment on the offending line. This is for a specific item and only this one. If more code is written in the same way and Globalyzer should not detect the issue, a Rule Set modification may be better.
Let's look at a simple line of code where Globalyzer detects a candidate issue, here another embedded string:
String marker = "Marker 15";
By adding the //$NON-NLS-L$
comment at the end of the line, Globalyzer will now ignore the issue:
String marker = "Marker 15"; //$NON-NLS-L$
Since the code is pushed to the source control system, the next time a Workbench user or a script gets the latest code, Globalyzer will ignore the issue.
The default list of Globalyzer Tags is:
$NON-NLS-L$
: Ignores any issue on the line$NON-NLS-[number]$
: Ignores one particular issue on the line. For instance $NON-NLS-2$ ignores the second issue on the lineGLOBALYZER_TODO
: Indicates something to be done in i18n. It overrides/detects the line and sets a TODO status to it.GLOBALYZER_IGNORE_NEXT_LINE
: Ignores any issue on the line after this commentGLOBALYZER_START_IGNORE
: Ignores a block of issues until the GLOBALYZER_END_IGNORE tag commentGLOBALYZER_END_IGNORE
: Stops ignoring a block of issues, working with GLOBALYZER_START_IGNORE
Globalyzer Workbench Prediction
Predictions are used to differentiate true issues from false positive ones.
When using the Globalyzer Workbench, you may want to change the prediction of some issues locally to your instance of the Workbench.
This will update the database tracking issues on your system; It does not affect what other users or system detect.
This helps organize the issues locally quickly, for instance when externalizing strings with the Workbench. You can mark the prediction of issues as TRUE(T)
if you
know the issue is a true positive, FALSE(F)
if you know the issue is a false positive, or PENDING(P)
if you are unsure but you want to indicate
that the issue needs further evaluation.
To change the prediction of an issue, select it, click the right mouse button, and click on the menu item. In the example below, the prediction of the issue will be 'F' locally.
Note: After marking predictions, you can export a Prediction Report with the markings (or you can export your entire Globalyzer project with the result data). Any user or system can then import the Prediction Report (or import the entire Globalyzer project) and it will set the result data prediction in the local database where the import happened.
In the Workbench, you designate an issue as a False Positive by marking its prediction False
.
After marking issues False
, you then export a Prediction Report. This saves the markings to a file that other Workbenches can import
and that Lite can use when generating reports displayed on the Dashboard. The Dashboard knows to ignore issues with False
predictions;
those issues won't appear on the Dashboard.
So false positive information moving from the Workbench to the Dashboard is fairly seamless, and uses the Prediction Report. No special explicit connection between the
Workbench and Dashboard is necessary.
Going the other direction, from the Dashboard to the Workbench, does require an explicit connection between the Workbench and the Dashboard. On the Dashboard, you can set issues to False Positive and you can reopen issues that once were False Positive and make them active again. Reflecting these Dashboard changes back to the Workbench requires first setting up a connection between the Workbench and the Dashboard:
1.Connecting to Lingoport Dashboard
In the Workbench, click on the Window menu and choose Preferences from the dropdown menu. Select Globalyzer=>Lingoport Dashboard Settings.
Specify the Server URL. Then, there are two ways to login. The default setting is use a Username/Password. If you want to login with a token, check "Use token to login". Click the Apply button to apply your settings and the Globalyzer Workbench will try to establish a connection to the Lingoport Dashboard with your credentials.
2. Set Dashboard Project Key for Workbench project
Click on the Project menu and choose Manage Globalyzer Projects from the dropdown menu. Select the Globalyzer Project you want to synchronize, then modify the Project Key at Dashboard value. Please note that you should enter the Project Key here instead of the Project Name; you can find the Project Key on the Dashboard.
Once the connection is established, the Workbench can synchronize against the Dashboard by either selecting the Project menu or by right-clicking in Scan Results and then selecting Reload Dashboard False Positives.
This action will identify all the False Positives on the Dashboard and update your Scan Results accordingly. False Positive issues from the Dashboard will be
given DFP
prediction values in the Workbench.
Confirmation information is written to the Workbench Console.
Feedback to Lingoport
For issues you strongly believe are incorrectly detected in the default rule set, feel free to select those lines, press the right mouse button, and select "Report as False Positive". This will send an email with the lines of code and the issues that you feel should not be detected.
This will help Lingoport refine the default rule set for future releases.
Machine Learning
Machine Learning prediction is a Globalyzer Workbench and Globalyzer Lite feature that helps users more quickly identify the real issues in their source code. We suggest applying Machine Learning as a follow-up step to scanning with Rule Sets. It helps to determine which candidate issues are indeed i18n issues.
Please see the Machine Learning page for detailed instructions on how to use Lingoport's Machine Learning implementation.