JMRI: Code Internationalization
Author: Bob Jacobsen
Keywords: JMRI technical code I18N internationalization

# JMRI Code: Internationalization

The JMRI libraries are intended to be usable world-wide. To make this possible, they make use of the "internationalization" features built into the Java language and libraries.
This page discusses how the JMRI libraries handle internationalization.

## Use of Locales

JMRI uses the default Locale for localizing internationalization information. That means that JMRI will present its user interface in the language Java has defined as the default for that computer.

Locales are specified by a Language, and optionally a Country. The Language is a two letter lower-case code; the Country is a two letter upper case code. "en" is English, "fr" is French, "de" is German, and "de_CH" is German as spoken in Switzerland.

When Java looks for resources (see below), it searches first for a file with the complete current Locale at the end of it's name (e.g. foo_de_CH.properties). If that fails, it tries for a file ending in just the current Locale's language: foo_de.properties. And if that fails, it goes to the defaults with no suffix: foo.properties. A similar mechanism is used within XML files.

By installing appropriate files and allowing the user to select a default Locale (as part of the Advanced Preferences), we can customize the JMRI® program to different countries and languages.

## Use of Resource Bundles

The text for menus, buttons and similiar controls is for the most part contained in property files, which are accessed via the Resource Bundle mechanism of java.util.

For example, the property file that's used to configure the Roster panel contains lines like:

    FieldRoadName = Road Name:

To the left of the equal sign is the Resource Name that the program uses to refer to the string; to the right of the equals sign is the String that will be displayed.

By convention, resource names for GUI elements start with one of

1. Field - for a visible field, e.g. label, on the GUI
2. Button - for a GUI button
3. Menu - the name of a top-level menu
4. MenuItem - an item in a menu (may be a nested item)
5. ToolTip - contents of a tooltip
6. Error - for an error message displayed as part of the GUI

Other resources are named so as not to conflict with these.

Many standard names for buttons, objects, colors etc. are grouped in one place, from which they are available in all tools, tables and JMRI apps. The highest level Bundle is called NamedBeanBundle, located in the jmri package. A next level is inside the jmri.jmrit (Tools) directory, where the Bundle.properties bundle contains keys common for all tools, such as the names of colors and other Tools interface strings.

## Adapting to a new language

The primary steps to adapt JMRI to a new language are:

1. Create new versions of the `.properties` files to change the language of the GUI controls.
2. Translate the XML files for decoders, programmers and configuration.
3. Translate the Help files and other web pages.

Get a clean copy of the source code from the JMRI code repository. (For more info on doing that, please see the page on getting a copy of the code.)

### Translating Properties Files

If they don't exist already, start by making copies of the properties files with suffix for your new locale. On a Mac OS X or a Unix machine, this would be:

      cd java/src/apps
      cp AppsBundle.properties AppsBundle_xy.properties

The easiest way to find the proper suffix letters for a Language and Country is to set the JMRI module to your particular Language via the Advanced Preferences ⇒ Display ⇒ Language tab, quit and restart the program, and then look at the suffix that the JMRI module displays on the startup screen/main window (in the bottom line, between the brackets after the Java version). You can also check the official list of languages (first part of the suffix) and list of countries/regions (optional second part of the suffix).

You then edit the language-specific files to enter text in your own language.

The lines in the file that contain things like `$Release: $;` are a vestige of older version control systems; they can be ignored or deleted.

There are several .properties files that are used for internal control, and should not be translated. These are marked by a comment at the top of the file. An example is the `apps/AppsStructureBundle.properties` file.

You are advised to start a new language translation by completing the highest level bundles, starting with the NamedBeanBundle in the src/jmri directory end working your way down the folder hierarchy, of course following your personal interest in certain modules. By following this order, you will see your initial work all the way down inside the user interface for tools such as Panels, Operations and Signals.
Thanks to the hierarchy of bundle properties files, we maintain consistency across the user interface of the different parts. If you come across a spot where your language just doesn't work, please leave a note with the developers so we can help you by adding a specific key, or even splitting the tree.

### Translating XML Files

The `xml/config/parts/jmri/` folder contains additional text strings to translate for programmers etc. Just as in decoder xml files, translated strings are inserted as `<name xml:lang="da">Your Translation</name>` elements in each node. We provide a list of editors to effectively work on these files.

### Check your Work

To check your work:

1. Rebuild your copy of the program, i.e. with your IDE or however you're doing that
2. Start the JMRI® application and select "Preferences" from the Edit menu;
3. Click the Display tab at the left, and the Locale tab in the right hand pane;
4. Select your language from the drop-down box (mind that once you run JMRI in another Locale, the list of languages will also be translated to thta language, changing the order for e.g. "Anglais" instead of "English";
5. Click "Save" at lower left, quit and restart;
6. You should immediately seen the items you've translated.

If there's a problem at this point, check to see what language is listed on the application startup screen. Is it showing the same suffix (e.g. _fr or _cs_CZ) as you gave to your files? The suffix JMRI® uses is determined by the Locale you selected in the preferences above.

To make your work available to other JMRI users, please share it with us contributing via GitHub. By using a GitHub Pull Request, it's easy for us to merge your new and/or changed files into the code repository. If anything goes wrong, please don't hesitate to ask for help with this.

#### Non-Roman characters

Languages that involve non-roman letters require some extra care. The .properties files must contain only ISO 8859-1 characters. If you want to use Unicode characters, these must be manually escaped. Please see the Java internationalization FAQ for more info on how to include those characters in your .properties files, particularly the question on "How do I specify non-ASCII strings in a properties file?".

An example is the java/src/apps/AppsBundle_cs.properties file, which contains diacritical letters for the Czech translation.

The "native2ascii" tool can help with this by converting special characters to the right sequences, but you must run your files through it before submitting them to JMRI. Special characters are somewhat machine-specific, so to make they're properly handled, they have to be converted to ISO 8859-1 sequences on your computer before submission.

### Translating XML files

XML files can also be internationalized. There are examples in the decoder definition directories. Look for elements with an xml:lang attribute. Basically, you create additional elements with that attribute to specify the language used:

          <variable CV="6" default="22" item="Vmid">
            <decVal max="64"/>
            <label>Vmid</label>
            <label xml:lang="fr">Vmoy</label>
          </variable>

In the XML files, the 'item' attributes have to stay untranslated, as does the entire xml/names.xml file.

There are XSLT transforms that can insert default language elements into the files. They still have English content, but it's perhaps easier to just translate English text than to edit in new XML elements, make sure the structure is correct, etc. For more information, see the xml/XSLT/I18N file or ask on the jmri-developers list.

### Translating Help files

(This has only been done once, so these instructions may not be complete)

The English help files are found in the help/en directory. If you want to create a complete set of files:

- Create a copy of the existing files from the help/en directory in a new help/LL directory, where LL is the Language code for your language, e.g. help/fr. (Please be careful doing this directly in GitHub, and ask a developer for help if needed)
- Rename the `help/fr/JmriHelp_en.hs` file in the copy you just created to `help/fr/JmriHelp_fr.hs`
- Edit the `help/fr/format.xsl` to create a `<HTML LANG="fr">` tag.
- Translate all the .shtml files inside the help/fr/ directory. Do not translate any `.xml` and `.jhm` files or the `web*.shtml` files in the top help/LL/ directory, as they are automatically produced.

## Internationalization for Developers

For internationalization to work, you have to do a few things in the code you write. Some web references on how to do this:

- Java Internationalization main page
- Sun internationalization tutorial (highly recommended)

Note: Those are Java 6 links. There are useful advanced features in Java 7 and Java 8.

JMRI is moving toward a set of conventions on how to structure and use the large amount of I18N information required. You'll still find code with older approaches, but you should write new code using the new conventions described below.

JMRI resource bundles are organized in a hierarchical tree. For example, code in the `jmri.jmrit.display` package may find a resource within a bundle in the `jmri.jmrit.display` package, the `jmri.jmrit` package or finally the `jmri` package. As a special case in this, the `apps` package is viewed as being below the `jmri` package itself, so code in the `apps.` tree also can reference the `jmri.` package.

Cross-package references, e.g. between `jmri.jmrit` and `jmri.jmrix`, are discouraged and existing ones are being removed.

Access is via a Bundle class local to each package. A typical one is `jmri.jmrit.Bundle`. It provides two key methods you use to access (translated) resource strings:
`static String getMessage(String key)
static String getMessage(String key, Object... subs)`

The first method provides direct access to a string via:
`String msg = Bundle.getMessage("Title")`.

The second method is used to insert specific information into a message like:
`System name LT1 is already in use`

Here "LT1" can't be in the .properties file, because it's only known which name to display when the program is running. Different languages may put that part of the message in different places, and supporting that is important. That's handled by putting a placeholder in the message definition:

    Error123 = System name {0} is already in use

(You can have more than one insertion, called {1}, {2}, etc)

Next, format the final message by inserting the content into it:

      String msg = Bundle.getMessage("Error123", badName);

The first argument is the message key followed by one or more strings to be inserted into the message. (This is better than creating your own output string using e.g. String.format() because it allows the inserted terms to appear in different orders in different languages.)

Different languages may need a different number of lines to express a message, or may need to break it before or after a particular value is inserted. It's therefore better to use "\n" within a single message from the properties file to create line breaks, rather than providing multiple lines in the code itself.

Some parts of JMRI remain English only due to our developer population. In particular, comments and variable names in the code should remain in English, as should messages sent to the logging system. Keys for the translation bundles should also stay in English. In the Java code, these strings can be marked with a "`// NOI18N`" comment at the end of the line. If needed, put that after another comment:

               Sensor thisSensorVariableNameIsInEnglish;

               String message = "THAT_ONE_MESSAGE";  // NOI18N
               JLabel jl1 = new JLabel(Bundle.getMessage(message));

               JLabel jl2 = new JLabel(Bundle.getMessage("LABELKEY"));  // NOI18N

               log.debug("The process failed account of user error"); // NOI18N

               // This comment is in English and need not be annotated w.r.t. internationalization

### Adding a new Bundle

If your package does not already have a Bundle class, you can add it by:

- Copy the Bundle class `java/src/jmri/jmrit/Bundle.java` into your package as `java/src/jmri/mypackage/Bundle.java`
- Edit the new file in three places:
  1. The 'package' statement at the top should list your package;
  2. The 'class ... extends' should refer to the Bundle class directly above your package;
  3. The assignment to the 'name' variable should be the name of your local bundle, by convention "jmri/mypackage.Bundle".
- Create a new `Bundle.properties` file in your package directory to hold your default properties strings.
- Ideally, you'll add a copy of `java/test/jmri/jmrit/BundleTest.java` to your JUnit test directory to check that your strings are working:
  Copy `java/test/jmri/jmrit/BundleTest.java` to `java/test/jmri/mypackage/BundleTest.java` and then edit the "package" statement in that file to point to your package, adding a few of your strings for testing (including ones you reference from parent bundles, if any), and including a reference in your PackageTest class.

#### Older code

Older code typically references the bundles directly:

      java.util.ResourceBundle.getBundle("jmri.jmrit.beantable.LogixTableBundle");

The getBundle argument is the complete package name (not file name) for the properties file this class will be using. You can reference more than one of these objects if you'd like to look up strings in more than one properties file.

You can then retrieve particular strings like this:

    java.util.ResourceBundle.getBundle("jmri.jmrit.beantable.LogixTableBundle").getString("ButtonNew");

We no longer recommend defining a class-static variable to hold the reference to the Bundle object, as this ends up consuming a lot of permanent memory in a program the size of JMRI.
Go ahead and call the `getBundle()` each time, it's fast because it works through a weakly-referenced and garbage-collected cache.

### XML Access

Second, you have to retrieve XML elements and attributes properly. The jmri.util.jdom.LocaleSelector provides a getAttribute(...) method that replaces the JDOM getAttribute element when the content of the attribute might have been internationalized. You use it like this:

    String choice = LocaleSelector.getAttribute(choiceElement, "choice")

where "choiceElement" is a JDOM Element object containing a (possibly translated) "choice" attribute. "Null" will be returned if nothing is found.

### Numbers

The number "10*10*10+2+3/10" is written in different ways in different places: "1002.3", "1,002.3", "1.002,3" and perhaps other ways.

JMRI provides a helpful utility for handling this on input:

       double d = jmri.util.IntlUtilities.doubleValue("1,002.3");
       float  f = jmri.util.IntlUtilities.floatValue("1,002.3");

Note that this can throw a `java.text.ParseException` if the input is unparsable, so handling that is part of your user-error handling.

For output:

       String s = jmri.util.IntlUtilities.valueOf(1002.3);

Note: You should still store and load values in XML files in Java's default coding, without using these formatting tools. That way, the files can be moved from one user to another without worrying about whether they are using the same Locale.

### Testing

You should check that you've properly internationalized your code. We provide a tool for doing this which creates an automatically translated version of your properties files, following the ideas of Harry Robinson and Arne Thormodsen (Their paper on this is recommended reading!). To use it:

- Make sure your code compiles and builds OK in your IDE. We'll be modifying the compiled version.
- Run the "translate.sh" script in your java/ build directory. This creates new, temporary .properties files in the classes/ directory tree. You will have to redo this every time the classes/ tree is removed by e.g. "ant clean" or a clean IDE build.
- Delete the JMRI Preference file, or edit it to remove the GUI definition line.
- Run DecoderPro via "ant locale", which starts the DecoderPro program using the new .properties files.

If all is well, all the message text will have been translated to UPPER CASE. Anything you wrote that remains in lower case has not been completely internationalized.

### Bundle Keys Report

BundleKeysReport.py, located in the scripts (not jython) directory, is used to analyse the bundle keys within a property file. The primary function is to identify unused keys. The script is run using PanelPro Scripting {Old: Panels} ⇒ Run Script.... The output from the script is written to the Script Output window. The run time will vary based on the number of keys to be checked along with the position in the source hierarchy. It will range from several to many seconds.

Once a property file, normally the default/English file, is selected, all of the classes within the package are scanned for each key in the file. if there are more packages below the initial one, their classes are also scanned. This covers the bundle hierarchy. Note: It is possible to get false positive matches when a class is using a matching key but the class is using a private property file.

After the unused key list is built, the entire source tree is scanned for external references to the selected property file. If the class containing the reference uses any of the unused keys, those keys are removed from the unused key list. The jython directory is also scanned for external references.

After the scanning is done, a dialog box prompts to save the unused key list. If desired, the list will be written with the selected location and name. The default location will be the User Files Location.

The final dialog box asks if the property files should be updated. If Yes is selected, all of the property files in the bundle set are backed up. Each file is then scanned for the unused keys. When one is found, the line is updated with #NotUsed as a comment. If testing reveals that the key is actually required, the comment can be removed. Note: If the source tree is managed by Git, the backups will be included in the current branch. Either move the backups or don't select them when doing a commit.

### Class Keys Report

ClassKeysReport.py, located in the scripts (not jython) directory, is used to identify the bundle keys used by a class. The script is run using PanelPro Scripting {Old: Panels} ⇒ Run Script.... The output from the script is written to the Script Output window.

When the script is started, a file selection dialog is displayed. Select either a Java class file or a Java package directory. If a directory is selected, all of the *.java files within the directory will be processed. The Bundle.java file is excluded.

For each file, the script scans for Bundle.getMessage( and getString(. The first word after the parenthesis is returned as the bundle key. A word is defined as the characters a-z, A-Z, 0-9 and underscore. If the first word is a Locale reference, the second word is returned.

Here is a typical output line:

       783, Search Type = Local, Key Type = Variable, Key = 'titleId',

             Text = addLogixFrame = new JmriJFrame(rbx.getString(titleId));

Field List

- The source code line number
- Search Type
  - Local: getString, such as rbx.getString()
  - Bundle: getMessage()
- Key Type
  - String: A word wrapped in double quotes
  - Variable: A plain word
- The key
- The source code line

Only the variable key types are displayed in the script output window.
Note: A key that contains non-word characters will be truncated and assigned the Variable key type.

When the script is done scanning, it provides an option to export the entire key list to a CSV file.

### Translators Helper

TranslatorsHelper.jar is a simple tool for translators. The program searches for all `.properties` files in all used languages. It then counts the untranslated keys for a specific language and file. This will help translators to find untranslated keys. The source code of the program is available here https://github.com/sidlo64/TranslatorsHelper.

The program can be started from the command line with the command:

`java -jar TranslatorsHelper.jar`

After startup enter the directory with the source code `$your-ide-dir$/JMRI/java/src` in the Start directory field. Click the Run button to start processing.

In the Languages tab, the program displays all found languages and the number of translated keys.

In the Files tab it is possible to select a specific language. The table below the selection shows all found .properties files and the number of translated keys.