Scanning and Filtering FAQ

Can I use Globalyzer to look for issues in my code that have nothing to do with internationalization?

Yes. In fact this is quite common. One of the added values that our customers use Globalyzer for is as a repository for regular expression searches, which they can name, describe and share among their company’s developers. See Rule Sets and scroll down to General Pattern Scan Rules for more details.

Is there support for detecting UI alignment problems?

This would be handled adding new General Pattern Rules to your Rule Set that detect fixed widths.

What is the difference between the “Ignore” and “Invalid” Status?

These status choices can have any meaning, and are just a way to further break up the scan results. That said, we think that the Invalid status should be used in the cases where the scan detection is not correct. For example, say Globalyzer’s Embedded String scan found a detection that wasn’t a UI embedded string (for example, a programmatic string). In this case, you would set the result to the Invalid status. The Ignore status, on the other hand, would be used for embedded strings that do appear in the UI but that you don’t translate, for example, file extensions or program names. The TODO is another status that you could apply to scan results that you want to deal with at a later time. In the case of the Locale-Sensitive Method scan, you could apply the Ignore status to results where the method usage is not an issue and use the TODO status if you want to look at the method later.

How does Globalyzer determine whether or not an embedded string is an internationalization issue?

During the Embedded String scanning process, Globalyzer will detect all strings in the source code. Then, the Globalyzer filters are used to eliminate the programmatic strings, or strings that would not require translation. So, the scanning work is primarily in filtering out strings, rather than in detecting them.

How are i18n issue priorities set?

Each active issue has a priority. The priority ranges between 1 and 5, 1 being the highest - i.e. the most likely to be an internationalization issue. In addition, Strings can be marked as 'Concatenated', or priority 0. For Locale-Sensitive Methods, Static File References, and General Patterns, the priority is configured in the detection rules of the rule set. For Embedded Strings, priorities can be configured in retention rules, such as String Content Patterns, String Method Patterns, and String Variable Patterns.

If a string is detected by these rules, the configured priority is assigned to the issue. If not, Globalyzer assigns a priority based on the content of the string. For example, a string containing words found in the dictionary will get a higher priority (more likely to be displayed to the end user and therefore more likely to require externalization) than one with words not found in the dictionary. Other considerations for assigning priorities to Embedded Strings include capitalization and punctuation found in the string. Priorities help focus efforts on the most important active issues.

String Priorities

For strings, here is the algorithm we use to determine priority:

Priority: 0 / Concatenation

Any string which is not filtered and is concatenated is marked as 0/Concatenated.

Priority 1:

• if dictionary filtering is ON:

Note: Dictionary filtering is turned on by default (see Window->Preferences->Globalyzer in the Globalyzer Workbench).
Note: Globalyzer comes with its own dictionary. To add words to the dictionary, create a Company dictionary that is used in addition to the Globalyzer dictionary. See here for more information.

   all words found in dictionary (one or more words)
   and first word character is uppercase (and second character of first word is not uppercase) 
   and string doesn't contain non word characters //Are these alpha numeric and special characters?

• if dictionary filter is OFF:

   first word character is upper case (and second character of first word is lowercase) 
   and string doesn't contain non word characters

Priority 2 (not Priority 1 and):

• if dictionary filtering is ON:

   all words found in dictionary 
   and first word character is lowercase (i.e. first letter is lowercase) 
   and number of terms is >= 2 // what is number of terms means, number of contiguous words?
   and string doesn't contain non word characters

   OR

   all words found in dictionary 
   and first word character is uppercase (and second character of first word is lowercase) 
   and string contains non word characters

• if dictionary filter is OFF:

   first word character is lowercase (i.e. first letter is lowercase) 
   and number of words is >= 2 
   and string doesn't contain non word characters

   OR

   first word character is uppercase (and second character of first word is lowercase) 
   and string contains non word characters

Priority 3 (not Priority 1, 2 and):

• if dictionary filtering is ON:

   all words found in dictionary (including one word phrase) 
   and first word character is lower case (i.e. first letter is lowercase) 
   and string contains non word characters

• if dictionary filtering is OFF:

   first word character is lower case (i.e. first letter is lowercase) 
   and string contains non word characters

Priority 4 (not Priority 1, 2, 3 and):

• if dictionary filter is ON:

   at least one word found in dictionary
   and number of words is 2 or more

Priority 5: not Priority 1, 2, 3, 4

Locale-Sensitive Methods Priorities

For Locale-Sensitive Methods, General Patterns, and Static File References, our general approach is to assign it a priority 1 if the application could break if the i18n issue isn't addressed. For example, an encoding issue where you would see character corruption. We also assign priority 1 to methods that are likely to have an i18n issue, such as date/time/number formatting methods. We assign as priority 2 or 3 issues which could be addressed later, such a sorting or font issues. These priorities are defaults which can be modified in the rule set.

How should I filter false positive issues?

See False Positives

Two of Globalyzer's main benefits are first in detecting possible internationalization (i18n) issues, and then in helping you hone down that list to only issues that will actually need code modifications to fix. There are three approaches that Globalyzer provides to address false positive issues in your Scan Results:

1. Add filter rules to your Rule Set so that the next time you scan, Globalyzer will filter out those false positive detections. Since Rule Sets can be shared among team members and applied to any source code, this is a way to make a large scale effect with little effort and is our first recommendation for filtering Scan Results.

2. Change the Scan Results Status of false positive issues to be Invalid or Ignore. The Status settings are stored with the scanning results in Globalyzer's Client database on your local machine, so as long as you don't delete your scan results or modify the source code file significantly, the Status settings will be maintained during subsequent scans. We recommend this approach when there are specific results that you need to filter that don't lend themselves to Rule Set filters, and when developers prefer that you do not modify source code.

3. The last approach is to use one of the Ignore Comment options, which modifies the source code, and so is the most permanent resolution. Regardless whether a Rule Set filter is modified or Scan Results are deleted, or the source code file is significantly modified, Globalyzer will detect the ignore comment in the source code and skip over one or more lines of code that contain false positive i18n issues.

In addition, Globalyzer's Dictionary, enabled by default in the Window->Preferences->Globalyzer page, helps to filter and prioritize the Embedded Strings that are detected. For example, strings will be filtered if there are no matching Dictionary words. Conversely, depending on the number of matching versus non-matching words as well as other applied heuristics, Globalyzer will set the priority of each Scan Result, which aids in determining which strings will most likely be translatable strings.

Can you tell me more about inserting comments in my source code so that Globalyzer does not detect programmatic strings?

The following comments can be used to tell Globalyzer some issues are not to be detected:

1. Use Eclipse style comments: select the rows that you want to ignore in the Scan Results table; Then right-click and choose Ignore this Issue. Globalyzer will insert comments of the form $NON-NLS-[n]$ at the end of the line, where "n" refers to the nth issue on the line. For example, if there were 2 embedded strings on the line that you wanted to ignore, Globalyzer would insert: $NON-NLS-1$ $NON-NLS-2$ for the two times that you chose Ignore this Issue.

4. Ignore all the issues on a line, choose Ignore this Line. Globalyzer will insert comments with $NON-NLS-L$ comments at the end of the line. Subsequent scans will ignore the lines with $NON-NLS-L$ comments.

1. Insert a comment in the source code that indicates the next line does not require scanning. The comment itself is customizable in Globalyzer's project properties setting. By default it is GLOBALYZER_IGNORE_NEXT_LINE.

2. Skip scanning a block of code by using a comment to start and a comment to end the "skipped block". These comment commands are also customizable. By default, the keywords are: GLOBALYZER_START_IGNORE and GLOBALYZER_END_IGNORE.

Note: Globalyzer Workbench also has a way to ignore issues without modifying the source – using the Status setting in the Scan Results view. A Status on the Workbench is local to that Workbench instance. It is no seen by Lite for instance. To move the Status to the Dashboard, use the Dashboard False Positive Workbench function (See False Positives ). As long as the files don’t change too much, the Status will keep track of the issues. If you want to share your Status settings with other team members, you may choose to

• export your Globalyzer Project (with Scan Results checked): they will need to import the project. You will also need to make sure that the Globalyzer Server Rule Sets used by the project are set to shared so that they're available to the entire team.
• Use Dashboard False Positives to centralize all False Positives on the Dashboard. See the False Positives Synchronization section in the False Positives page.

Could you provide some sense of scanning performance?

Performance will depend on the machine running the scans. Lingoport performance tests are geared towards a Continuous Globalization process where a Jenkins type system executes scanning jobs. The following gives a performance benchmark for a specific machines on the two major operations when running in a Continuous Globalyzer:

• Login in
• Running the scans, with or without rule sets reload

Benchmark System Info:
   Operating System Name: Windows 8.1
   Operating System Type: amd64
   Operating System Version: 6.3
   Processor Identifier: Intel64 Family 6 Model 58 Stepping 9, GenuineIntel
   Number of Processors: 4
   Available processors (cores): 4
   Free memory (bytes): 79497416
   Maximum memory (bytes): 1407713280
   Total memory (bytes): 94896128
   Number of Rule Sets: 10

Login Benchmark: 559 milliseconds
Login Benchmark: 324 milliseconds
Login Benchmark: 336 milliseconds
Login Benchmark: 313 milliseconds
Login Benchmark: 307 milliseconds
Login Benchmark: 306 milliseconds
Login Benchmark: 313 milliseconds
Login Benchmark: 300 milliseconds
Login Benchmark: 303 milliseconds
Login Benchmark: 290 milliseconds


Execute Java Scan Benchmark (927 lines of code, 9 files, 190 active results):
Reloaded rule sets
Executing java scan: 2073 milliseconds
Executing java scan: 746 milliseconds
Executing java scan: 557 milliseconds
Executing java scan: 553 milliseconds
Executing java scan: 533 milliseconds
Executing java scan: 541 milliseconds
Executing java scan: 520 milliseconds
Executing java scan: 513 milliseconds
Executing java scan: 524 milliseconds
Executing java scan: 516 milliseconds
Reloaded rule sets
Executing java scan: 652 milliseconds
Executing java scan: 574 milliseconds
Executing java scan: 526 milliseconds
Executing java scan: 493 milliseconds
Executing java scan: 488 milliseconds
Executing java scan: 499 milliseconds
Executing java scan: 486 milliseconds
Executing java scan: 481 milliseconds
Executing java scan: 475 milliseconds
Executing java scan: 470 milliseconds
Reloaded rule sets
Executing java scan: 611 milliseconds
Executing java scan: 478 milliseconds
Executing java scan: 465 milliseconds
Executing java scan: 472 milliseconds
Executing java scan: 464 milliseconds
Executing java scan: 483 milliseconds
Executing java scan: 469 milliseconds
Executing java scan: 455 milliseconds
Executing java scan: 459 milliseconds
Executing java scan: 459 milliseconds

How can I scan code written in programming languages Globalyzer does not support out of the box?

Globalyzer provides a set of internationalization rules in support of the main programming languages. In case your code is written in a language that is not part of the out of the box rules, you can do the following:

• Call Lingoport to include a new set of rules for you: This may or may not be a consulting effort by Lingoport
• Find a supported programming language whose constructs are close to your own and create your own rule set. To do so, please visit the Scanning Unsupported Languages page.

How can I add words to the dictionary used when scanning?

If the preference Filter Strings without Dictionary Words is checked (see Window->Preferences->Globalyzer), Globalyzer will automatically filter strings that contain no dictionary words. That is, the string itself is parsed into words, and if none of the words are found in the dictionary, the string is filtered. If at least one of the words is found in the dictionary, the string is potentially active.

The dictionary is maintained by Lingoport. If there are words you would like added to the dictionary, you can create a Company Dictionary on the Globalyzer Server and add words to that. There is one Company Dictionary per company, so all users in the company will see the same dictionary.

To add words to the Company Dictionary, download the dictionary file from the Globalyzer Server, modify the file locally (one word per line), and then upload the dictionary file to the Server.

To use your Company Dictionary for scanning, download the dictionary file from the Server and place in your Lingoport directory (C:\Lingoport on Windows, $HOME/lingoport on Linux, and /Applications/Lingoport on Mac). When scanning, Globalyzer will consider words in both the Lingoport Dictionary and your Company Dictionary.