Globalyzer Workbench

General

Can Globalyzer Workbench help me build resource files and externalize strings from my source code?

Yes it can, and it will add great productivity to that long and tedious task. It is common for developers to be able to comfortably externalize 1,000 strings and more per day using Globalyzer Workbench. Globalyzer Workbench lets you see the string within your code and manages the whole externalization process. It is customizable, giving the developer complete control while aiding productivity.

How do I install Globalyzer Workbench ?

To install Globalyzer Workbench, you first need a Globalyzer account (see this FAQ). Once you have an account, you log into Globalyzer Server and at the bottom of your home page there is a link to download Globalyzer Workbench.

Globalyzer Workbench is a standalone Eclipse Rich Client Platform (RCP) application. In addition to many internationalization features, the Workbench benefits from Eclipse's support for integrating with source control, programming language plug-ins and any other plug-in to assist with your development.

Please see installation for complete Globalyzer Workbench download/installation instructions.

Is Globalyzer Workbench automatically (automagically) fixing my code? Is Globalyzer Workbench adding anything proprietary?

No, Globalyzer Workbench does not make changes without your knowledge and does not add any proprietary elements to your source code. However, once you do approve the fix for one or more internationalization issues, Globalyzer Workbench does have highly productive capabilities to speed up repetitive internationalization tasks, such as string externalization.

What does "nature" mean?

"Nature" is an Eclipse notion. Project natures act as tags on a project to indicate that a certain tool is used to operate on that project. When you create a project and add a Globalyzer Nature to the project, that project can have scans, etc.

In Globalyzer Workbench, after the string extraction, can a developer tell what string is associated with the string key other than opening up the resource file?

If you select an externalized string key in the source code, you can right click and select Globalyzer->Resources->Go to Resource Definition

It will bring up the resource file with that entry highlighted.

Also, if you set the preference to Window -> Preferences -> Globalyzer - Automatically Insert Externalized String Comment, when strings are externalized, a comment will be added to the source file with the text of the string that was externalized.

Can a developer customize the result list – for example, have the freedom to move entries to to-do list or ignore list, etc.?

Yes, scan result statuses can be changed. Select one or more scan results, right-click and select the new status.

Scanning and Filtering

Can I use Globalyzer to look for issues in my code that have nothing to do with internationalization?

Yes. In fact this is quite common. One of the added values that our customers use Globalyzer for is as a repository for regular expression searches, which they can name, describe and share among their company’s developers. See Rule Sets and scroll down to General Pattern Scan Rules for more details.

Is there support for detecting UI alignment problems?

This would be handled by adding new General Pattern Rules to your Rule Set that detect fixed widths.

What is the difference between the “Ignore” and “Invalid” Status?

These status choices have been deprecated, but can still be used, although there are different ways to do things.

These status choices can have any meaning, and are just a way to further break up the scan results.

That said, we think that the Invalid status should be used in the cases where the scan detection is not correct. For example, say Globalyzer’s Embedded String scan found a detection that wasn’t a UI embedded string (for example, a programmatic string). In this case, you would set the result to the Invalid status.

The Ignore status, on the other hand, would be used for embedded strings that do appear in the UI but that you don’t translate, for example, file extensions or program names.

The TODO is another status that you could apply to scan results that you want to deal with at a later time. In the case of the Locale-Sensitive Method scan, you could apply the Ignore status to results where the method usage is not an issue and use the TODO status if you want to look at the method later.

How does Globalyzer determine whether or not an embedded string is an internationalization issue?

During the Embedded String scanning process, Globalyzer will detect all strings in the source code. Then, the Globalyzer Embedded String Filtering Rules are used to eliminate the programmatic strings, or strings that would not require translation. So, the scanning work is primarily in filtering out strings, rather than in detecting them.

How are i18n issue priorities set?

Each active issue has a priority. The priority ranges between 1 and 5, 1 being the highest - i.e. the most likely to be an internationalization issue. In addition, Strings can be marked as 'Concatenated', or priority 0. For Locale-Sensitive Methods, Static File References, and General Patterns, the priority is configured in the detection rules of the rule set. For Embedded Strings, priorities can be configured in retention rules, such as String Content Patterns, String Method Patterns, and String Variable Patterns.

If a string is detected by these rules, the configured priority is assigned to the issue. If not, Globalyzer assigns a priority based on the content of the string. For example, a string containing words found in the dictionary will get a higher priority (more likely to be displayed to the end user and therefore more likely to require externalization) than one with words not found in the dictionary. Other considerations for assigning priorities to Embedded Strings include capitalization and punctuation found in the string. Priorities help focus efforts on the most important active issues.

Globalyzer Workbench also allows the user to assign different priorities to issues.

String Priorities

For strings, here is the algorithm we use to determine priority:

Priority: 0 / Concatenation

Any string which is not filtered and is concatenated is marked as 0/Concatenated.

Priority 1:

• if dictionary filtering is ON:

Note: Dictionary filtering is turned on by default (see Window->Preferences->Globalyzer in the Globalyzer Workbench).
Note: Globalyzer comes with its own dictionary. To add words to the dictionary, create a Company dictionary that is used in addition to the Globalyzer dictionary. See How can I add words to the dictionary used when scanning? below for more information.

   all words found in dictionary (one or more words)
   and first word character is uppercase (and second character of first word is not uppercase) 
   and string doesn't contain non word characters (that is, string only contains letters, numbers, and/or white space)

• if dictionary filter is OFF:

   first word character is upper case (and second character of first word is lowercase) 
   and string doesn't contain non word characters

Priority 2 (not Priority 1 and):

• if dictionary filtering is ON:

   all words found in dictionary 
   and first word character is lowercase (i.e. first letter is lowercase) 
   and number of words is 2 or more 
   and string doesn't contain non word characters

   OR

   all words found in dictionary 
   and first word character is uppercase (and second character of first word is lowercase) 
   and string contains non word characters

• if dictionary filter is OFF:

   first word character is lowercase (i.e. first letter is lowercase) 
   and number of words is >= 2 
   and string doesn't contain non word characters

   OR

   first word character is uppercase (and second character of first word is lowercase) 
   and string contains non word characters

Priority 3 (not Priority 1, 2 and):

• if dictionary filtering is ON:

   all words found in dictionary (including one word phrase) 
   and first word character is lower case (i.e. first letter is lowercase) 
   and string contains non word characters

• if dictionary filtering is OFF:

   first word character is lower case (i.e. first letter is lowercase) 
   and string contains non word characters

Priority 4 (not Priority 1, 2, 3 and):

• if dictionary filter is ON:

   at least one word found in dictionary
   and number of words is 2 or more

Priority 5: not Priority 1, 2, 3, 4

Locale-Sensitive Methods Priorities

For Locale-Sensitive Methods, General Patterns, and Static File References, our general approach is to assign it a priority 1 if the application could break if the i18n issue isn't addressed. For example, an encoding issue where you would see character corruption.

We also assign priority 1 to methods that are likely to have an i18n issue, such as date/time/number formatting methods.

We assign as priority 2 or 3 issues which could be addressed later, such a sorting or font issues. These priorities are defaults which can be modified in the rule set.

How should I filter false positive issues?

See False Positives

Two of Globalyzer's main benefits are first in detecting possible internationalization (i18n) issues, and then in helping you hone down that list to only issues that will actually need code modifications to fix. There are three approaches that Globalyzer provides to address false positive issues in your Scan Results:

1. Add filter rules to your Rule Set so that the next time you scan, Globalyzer will filter out those false positive detections. Since Rule Sets can be shared among team members and applied to any source code, this is a way to make a large scale effect with little effort and is our first recommendation for filtering Scan Results.

2. Change the Scan Results Status of false positive issues to be Invalid or Ignore. The Status settings are stored with the scanning results in Globalyzer's Client database on your local machine, so as long as you don't delete your scan results or modify the source code file significantly, the Status settings will be maintained during subsequent scans. We recommend this approach when there are specific results that you need to filter that don't lend themselves to Rule Set filters, and when developers prefer that you do not modify source code.

3. The last approach is to use one of the Ignore Comment options, which modifies the source code, and so is the most permanent resolution. Regardless whether a Rule Set filter is modified or Scan Results are deleted, or the source code file is significantly modified, Globalyzer will detect the ignore comment in the source code and skip over one or more lines of code that contain false positive i18n issues.

In addition, Globalyzer's Dictionary, enabled by default in the Window->Preferences->Globalyzer page, helps to filter and prioritize the Embedded Strings that are detected. For example, strings will be filtered if there are no matching Dictionary words. Conversely, depending on the number of matching versus non-matching words as well as other applied heuristics, Globalyzer will set the priority of each Scan Result, which aids in determining which strings will most likely be translatable strings.

Can you tell me more about inserting comments in my source code so that Globalyzer does not detect programmatic strings?

The following comments can be used to tell Globalyzer some issues are not to be detected:

1. Use Eclipse style comments: select the rows that you want to ignore in the Scan Results table; Then right-click and choose Ignore this Issue. Globalyzer will insert comments of the form $NON-NLS-[n]$ at the end of the line, where "n" refers to the nth issue on the line. For example, if there were 2 embedded strings on the line that you wanted to ignore, Globalyzer Workbench would insert: $NON-NLS-1$ $NON-NLS-2$ for the two times that you chose Ignore this Issue.

4. Ignore all the issues on a line, choose Ignore this Line. Globalyzer Workbench will insert comments with $NON-NLS-L$ comments at the end of the line. Subsequent scans will ignore the lines with $NON-NLS-L$ comments.

1. Insert a comment in the source code that indicates the next line does not require scanning. The comment itself is customizable in Globalyzer's project properties setting. By default it is GLOBALYZER_IGNORE_NEXT_LINE.

2. Skip scanning a block of code by using a comment to start and a comment to end the "skipped block". These comment commands are also customizable. By default, the keywords are: GLOBALYZER_START_IGNORE and GLOBALYZER_END_IGNORE.

Note: Globalyzer Workbench also has a way to ignore issues without modifying the source – using the Status setting in the Scan Results view. A Status on the Workbench is local to that Workbench instance. It is not seen by Lite for instance. To move the Status to the Dashboard, use the Dashboard False Positive Workbench function (See False Positives ). As long as the files don’t change too much, the Status will keep track of the issues. If you want to share your Status settings with other team members, you may choose to

• export your Globalyzer Project (with Scan Results checked): they will need to import the project. You will also need to make sure that the Globalyzer Server Rule Sets used by the project are set to shared so that they're available to the entire team.
• Use Dashboard False Positives to centralize all False Positives on the Dashboard. See the False Positives Synchronization section in the False Positives page.

Could you provide some sense of scanning performance?

Performance will depend on the machine running the scans. Lingoport performance tests are geared towards a Continuous Globalization process where a Jenkins type system executes scanning jobs. The following gives a performance benchmark for a specific machines on the two major operations when running in a Continuous Globalyzer:

• Login in
• Running the scans, with or without rule sets reload

Benchmark System Info:
   Operating System Name: Windows 8.1
   Operating System Type: amd64
   Operating System Version: 6.3
   Processor Identifier: Intel64 Family 6 Model 58 Stepping 9, GenuineIntel
   Number of Processors: 4
   Available processors (cores): 4
   Free memory (bytes): 79497416
   Maximum memory (bytes): 1407713280
   Total memory (bytes): 94896128
   Number of Rule Sets: 10

Login Benchmark: 559 milliseconds
Login Benchmark: 324 milliseconds
Login Benchmark: 336 milliseconds
Login Benchmark: 313 milliseconds
Login Benchmark: 307 milliseconds
Login Benchmark: 306 milliseconds
Login Benchmark: 313 milliseconds
Login Benchmark: 300 milliseconds
Login Benchmark: 303 milliseconds
Login Benchmark: 290 milliseconds


Execute Java Scan Benchmark (927 lines of code, 9 files, 190 active results):
Reloaded rule sets
Executing java scan: 2073 milliseconds
Executing java scan: 746 milliseconds
Executing java scan: 557 milliseconds
Executing java scan: 553 milliseconds
Executing java scan: 533 milliseconds
Executing java scan: 541 milliseconds
Executing java scan: 520 milliseconds
Executing java scan: 513 milliseconds
Executing java scan: 524 milliseconds
Executing java scan: 516 milliseconds
Reloaded rule sets
Executing java scan: 652 milliseconds
Executing java scan: 574 milliseconds
Executing java scan: 526 milliseconds
Executing java scan: 493 milliseconds
Executing java scan: 488 milliseconds
Executing java scan: 499 milliseconds
Executing java scan: 486 milliseconds
Executing java scan: 481 milliseconds
Executing java scan: 475 milliseconds
Executing java scan: 470 milliseconds
Reloaded rule sets
Executing java scan: 611 milliseconds
Executing java scan: 478 milliseconds
Executing java scan: 465 milliseconds
Executing java scan: 472 milliseconds
Executing java scan: 464 milliseconds
Executing java scan: 483 milliseconds
Executing java scan: 469 milliseconds
Executing java scan: 455 milliseconds
Executing java scan: 459 milliseconds
Executing java scan: 459 milliseconds

How can I scan code written in programming languages Globalyzer does not support out of the box?

Globalyzer provides a set of internationalization rules in support of the main programming languages. In case your code is written in a language that is not part of the out of the box rules, you can do the following:

• Call Lingoport to include a new set of rules for you: This may or may not be a consulting effort by Lingoport
• Find a supported programming language whose constructs are close to your own and create your own rule set. To do so, please visit the Scanning Unsupported Languages page.

How can I add words to the dictionary used when scanning?

If the preference Filter Strings without Dictionary Words is checked (see Window->Preferences->Globalyzer), Globalyzer will automatically filter strings that contain no dictionary words. That is, the string itself is parsed into words, and if none of the words are found in the dictionary, the string is filtered. If at least one of the words is found in the dictionary, the string is potentially active.

The dictionary is maintained by Lingoport. If there are words you would like added to the dictionary, you can create a Company Dictionary on the Globalyzer Server and add words to that. There is one Company Dictionary per company, so all users in the company will see the same dictionary.

To add words to the Company Dictionary, download the dictionary file from the Globalyzer Server, modify the file locally (one word per line), and then upload the dictionary file to the Server.

To use your Company Dictionary for scanning, download the dictionary file from the Server and place in your Lingoport directory (C:\Lingoport on Windows, $HOME/lingoport on Linux, and /Applications/Lingoport on Mac). When scanning, Globalyzer will consider words in both the Lingoport Dictionary and your Company Dictionary.

Fixing i18n Issues FAQ

How do I get started on Fixing i18n Issues?

A set of short tutorial videos is available on the About Globalyzer page. These were made several years ago, but the process is still the same.

How can I tell if I am using the correct method call for a supported locale?

For many of its supported languages, Globalyzer will provide you with information and direction for refactoring methods and functions, giving examples and cross references when appropriate. It’s probably the largest on-line internationalization help system existing anywhere.

Will Globalyzer help me adapt my application to support Unicode?

Yes, as is necessary to your particular programming language. You will have a number of character encoding support choices.

How should I fix locale-sensitive Static File references?

Static Files that might contain internationalization issues are usually handled by the original files and the localized files in locale-sensitive directories. The code is then refactored to retrieve the file based on the user's locale. Some examples of locale-sensitive static files are images with text, help files, video files, HTML files, and xml files that contain displayable text.

Globalyzer Workbench's Static File Reference Scan will scan the source code for references to these types of static files. After you've filtered out the false positives, you'll need to refactor the references so that they use the locale to determine which static file to load.

Will Globalyzer Workbench automatically externalize embedded strings?

Globalyzer Workbench has the capability of externalizing an unlimited number of embedded strings with just a few keystrokes. However, once you select the strings you want to externalize, you may view each reported issue in a snippet of the code and exclude it from the externalization process as needed. While batch externalization is a huge time-saver in your internationalization process, we do recommend checking the strings carefully first.

Globalyzer Workbench Project FAQ

What is the difference between a Workbench Project and a Globalyzer Project?

A Workbench Project is an Eclipse Project. A Globalyzer Project is a set of scans. Ideally you would have one Workbench Project associated with one Globalyzer Project.

How do I create a Workbench Project associated with a Globalyzer Project?

1. Select File=>New to launch Eclipse's Project Wizard.

2. Choose the type of Workbench Project you want to create. For example, if your application is a Java application, you can choose Java Project. If you don't have a plug-in for your application's programming language or you're planning to build and run your application outside the Globalyzer Workbench, select General=>Project and click Next.

3. Enter a Project Name and source code Location in the New Project dialog.

4. Once the Eclipse Workbench Project is created, it will prompt you to create a Globalyzer Project and then Scans for the Project.

I have a Globalyzer Project without a Workbench Project; how did this happen and how can I create a Workbench Project?

This can occur for a variety of reasons:

• You've imported a Globalyzer Project through the Globalyzer Command Line Client.
• You've deleted the associated Workbench Project.

Follow these steps to associate your Globalyzer Project with a new Workbench Project:

1. Click on Project-->Manage Globalyzer Projects. This dialog lists all your Globalyzer Projects: Any Globalyzer Project that is associated with a Workbench Project will have a W icon next to its name.

2. Click on the Globalyzer Project that you want to associate with a Workbench Project.

Note that once you've associated your Globalyzer Project with a Workbench Project, you'll no longer be able to change its name or source code location until you break the association.

3. Click on the Associate button. This will bring up Eclipse's New Project wizard so that you can create the associated Workbench Project. Be sure to use the exact same Globalyzer Project name and source code location. Otherwise, the Eclipse Workbench Project won't be associated with the Globalyzer Project. Once you make the association, you'll see a W icon next to the Globalyzer Project's name. You'll also now be able to fully use Globalyzer features on the Project.

I have a Workbench Project without a Globalyzer Project, how did this happen and how can I create a Globalyzer Project?

This occurs when you've already set up an Eclipse Project for your application and now you want to use Globalyzer on it.

1. Right-click on the Eclipse Workbench Project in the Navigation pane.

2. Select Globalyzer=>Add/Remove Globalyzer Nature. This will create an associated Globalyzer Project and bring you into the New Scan Wizard.

What does "nature" mean?

"Nature" is an Eclipse notion. Project natures act as tags on a project to indicate that a certain tool is used to operate on that project. When you create a project and add a Globalyzer Nature to the project, that project can have scans, etc.

What are False Positives?

Globalyzer scans the source code to detect candidate issues based on rules. Given some code, a rule may detect a candidate issue. In one context, that candidate issue may need to be corrected when in another context, the same issue may not need any work and is a false positive.

Let's look at a couple of lines of Java code which creates a date string based on a hard coded date format using SimpleDateFormat:

SimpleDateFormat formatter = new SimpleDateFormat("MM/dd/yy");
String dateString = formatter.format(new Date()); 

Using the default Java rule set, this code will raise i18n issues. If the dateString variable is user facing, this code is indeed an issue to be fixed (for more information, check your rule set help pages).

However, if the dateString variable is used for internal purposes, such as in a support log file, it should not be modified and is a false positive.

This section describes different ways to handle false positives.

Modifying Rules

Since Rules are what detect or filter out issues, the most common way to handle particular programming styles is to modify the rule set applied to scan a project's code.

Let's look at a string detected by Globalyzer:

SpecialTB.write("New user created");

Globalyzer identifies the string New user created as a candidate embedded string which should be moved out of the code and into a resource file. However, let's assume the developer knows that SpecialTB.write() is a method that writes some logging information into a database for potential debugging purposes. The Rule Set needs to be modified based on that knowledge so that this particular line and all other strings passed to this call will be filtered out, i.e. will not be detected as a candidate issue. In this instance, a String Method Filter would be created as shown below:

Note: for the Java Rule Set only, we support Class or Variable Types, so you can filter strings when passed as a parameter to a particular object/method.

Any Globalyzer scan using this updated rule set will henceforth filter out the embedded string issue. Workbench instances and the nightly builds for instance would now filter out the issue.

You can also create and try new rules from the Globalyzer Workbench, and push the new rules to the server. To do so, right mouse click on an issue and select the Add Rule Sets Filters/Detection ... to show the dialog box.

Detailed Guide

Please visit the False Positive Filtering page for a detailed guide on creating false-positive filtering rules.

Globalyzer Comment Tags

When you want to filter out a specific item in the code, one way is to add a Globalyzer comment on the offending line. This is for a specific item and only this one. If more code is written in the same way and Globalyzer should not detect the issue, a Rule Set modification may be better.

Let's look at a simple line of code where Globalyzer detects a candidate issue, here another embedded string:

   String marker = "Marker 15";

By adding the //$NON-NLS-L$ comment at the end of the line, Globalyzer will now ignore the issue:

   String marker = "Marker 15"; //$NON-NLS-L$ 

Since the code is pushed to the source control system, the next time a Workbench user or a script gets the latest code, Globalyzer will ignore the issue.

The default list of Globalyzer Tags is:

• $NON-NLS-L$ : Ignores any issue on the line
• $NON-NLS-[number]$ : Ignores one particular issue on the line. For instance $NON-NLS-2$ ignores the second issue on the line
• GLOBALYZER_TODO : Indicates something to be done in i18n. It overrides/detects the line and sets a TODO status to it.
• GLOBALYZER_IGNORE_NEXT_LINE: Ignores any issue on the line after this comment
• GLOBALYZER_START_IGNORE : Ignores a block of issues until the GLOBALYZER_END_IGNORE tag comment
• GLOBALYZER_END_IGNORE : Stops ignoring a block of issues, working with GLOBALYZER_START_IGNORE

Globalyzer Workbench Prediction

Predictions are used to differentiate true issues from false positive ones. When using the Globalyzer Workbench, you may want to change the prediction of some issues locally to your instance of the Workbench. This will update the database tracking issues on your system; It does not affect what other users or system detect. This helps organize the issues locally quickly, for instance when externalizing strings with the Workbench. You can mark the prediction of issues as TRUE(T) if you know the issue is a true positive, FALSE(F) if you know the issue is a false positive, or PENDING(P) if you are unsure but you want to indicate that the issue needs further evaluation.

To change the prediction of an issue, select it, click the right mouse button, and click on the menu item. In the example below, the prediction of the issue will be 'F' locally.

Note: After marking predictions, you can export a Prediction Report with the markings (or you can export your entire Globalyzer project with the result data). Any user or system can then import the Prediction Report (or import the entire Globalyzer project) and it will set the result data prediction in the local database where the import happened.

In the Workbench, you designate an issue as a False Positive by marking its prediction False. After marking issues False, you then export a Prediction Report. This saves the markings to a file that other Workbenches can import and that Lite can use when generating reports displayed on the Dashboard. The Dashboard knows to ignore issues with False predictions; those issues won't appear on the Dashboard. So false positive information moving from the Workbench to the Dashboard is fairly seamless, and uses the Prediction Report. No special explicit connection between the Workbench and Dashboard is necessary.

Going the other direction, from the Dashboard to the Workbench, does require an explicit connection between the Workbench and the Dashboard. On the Dashboard, you can set issues to False Positive and you can reopen issues that once were False Positive and make them active again. Reflecting these Dashboard changes back to the Workbench requires first setting up a connection between the Workbench and the Dashboard:

1.Connecting to Lingoport Dashboard

In the Workbench, click on the Window menu and choose Preferences from the dropdown menu. Select Globalyzer=>Lingoport Dashboard Settings.

Specify the Server URL. Then, there are two ways to login. The default setting is use a Username/Password. If you want to login with a token, check "Use token to login". Click the Apply button to apply your settings and the Globalyzer Workbench will try to establish a connection to the Lingoport Dashboard with your credentials.

2. Set Dashboard Project Key for Workbench project

Click on the Project menu and choose Manage Globalyzer Projects from the dropdown menu. Select the Globalyzer Project you want to synchronize, then modify the Project Key at Dashboard value. Please note that you should enter the Project Key here instead of the Project Name; you can find the Project Key on the Dashboard.

Once the connection is established, the Workbench can synchronize against the Dashboard by either selecting the Project menu or by right-clicking in Scan Results and then selecting Reload Dashboard False Positives. This action will identify all the False Positives on the Dashboard and update your Scan Results accordingly. False Positive issues from the Dashboard will be given DFP prediction values in the Workbench.

Confirmation information is written to the Workbench Console.

Feedback to Lingoport

For issues you strongly believe are incorrectly detected in the default rule set, feel free to select those lines, press the right mouse button, and select "Report as False Positive". This will send an email with the lines of code and the issues that you feel should not be detected.

This will help Lingoport refine the default rule set for future releases.