JMRI: Structure of External System Connections
Author: Bob Jacoben
Keywords: JMRI technical system add

# JMRI Code: Structure of External System Connections

This page is about how JMRI connects to external systems, e.g. DCC systems.

There's a lot of variation within JMRI on this, so you'll have to go through any specific implementation. Specifically, older systems weren't always arranged this way, so existing code may not be a good example.

See also the Multiple Connection Update page.

## Code Structure

The code for a general type, like "LocoNet connections" or "NCE connections", should be gathered in a specific package right under `jmri.jmrix` e.g. `jmri.jmrix.loconet` and `jmri.jmrix.nce`. In the preferences dialog and `JmrixConfigPane` main configuration code, this level is called the "manufacturer selection". It provides a level of grouping, which we may someday want to use for e.g. providing separate updates for specific hardware, while still separating the system-specific code from the system-independent parts of JMRI.

Within that, the code should be separated further by putting specific hardware options into their own subpackages, for example

- `jmri.jmrix.loconet.locobuffer` vs `jmri.jmrix.loconet.locobufferusb` vs. `jmri.jmrix.loconet.pr3` vs. `jmri.jmrix.loconet.locormi`
- `jmri.jmrix.nce.serialdriver` vs. `jmri.jmrix.nce.usbdriver` vs. `jmri.jmrix.nce.simulator` vs. `jmri.jmrix.nce.networkdriver`

In the preferences, this is called the "connection mode" selection.

Additional subpackages can be used grouping various functions as needed. For example, Swing-based tools should go in their own swing subpackage or at a further level within the swing subpackage.

## Normal Operation

The key to normal operation (after start up and before shut down) is a `SystemConnectionMemo` object that provides all necessary access to the system connection's objects. For example, the `LocoNetSystemConnectionMemo` provides access to a number of LocoNet-specific objects and LocoNet-specific implementations of common objects. Although some of those (e.g. a SensorManager) might be separately available from the InstanceManager, accessing them from a SystemConnectionMemo allows you to find the consistent set associated with one specific connection of a multiple-connection setup, even when there are multiple connections of a specific type. There are also a few tools that work with the `SystemConnectionManager` objects themselves after obtaining them from the `InstanceManager`.

## Initialization

We don't directly persist the SystemConnectionMemo. This is partly for historical reasons, but it also reflects the level of abstraction: A SystemConnectionMemo is at the level of a "LocoNet connection" or a "NCE connection", and there's a lot of specific information below it to configure one of many possible such connections.

Instead, configuration of the connection is from the bottom up: From the most specific code up to the general. The "Adapter" object connects directly to the system, e.g. managing a serial link, and then builds up the objects that work with that link, including all the various type managers. This makes sense because the type of the connection is really specified via the type of that link and what's on the other end of it.

### "Simple" Initialization Sequence

This section describes the LocoNet implementation of the new (post-multiple) configuration system. This is similar for LocoBuffer, LocoBuffer-USB, PR3, etc connections, but we use the specific LocoBuffer-USB case for concreteness. This sequence picks up after the basic startup of the application itself, see the App Structure page for that.

There are several objects involved in startup:

- A `ConnectionConfigXml` object, created by the ConfigureXML system as part of reading the preferences. It drives the process.
- A `ConnectionConfig` object, registered so that a later store of the preferences will write out the right stuff
- An `Adapter` object of a very specific type, which handles both the connection to the system hardware, and (through its `configure()` method) the creation of the rest of the system.

The profile XML file contains a connection element that drives the configuration:

            <connection xmlns="" class="jmri.jmrix.loconet.locobufferusb.configurexml.ConnectionConfigXml"
                                    disabled="no" manufacturer="Digitrax" port="/dev/tty.usbserial-LWPMMU13"
                                    speed="57,600 baud" systemPrefix="L" userName="LocoNet">
                <options>
                    <option>
                        <name>CommandStation</name>
                        <value>DCS50 (Zephyr)</value>
                    </option>
                    <option>
                        <name>TurnoutHandle</name>
                        <value>Normal</value>
                    </option>
                </options>
            </connection>

Initialization proceeds through multiple steps (click on the diagram to expand it):

- An object of type `jmri.jmrix.loconet.locobufferusb.configurexml.ConnectionConfigXml` is constructed by the configurexml mechanism when the specific class is named by the file during the initial preference load at application startup.
- The ConnectionConfigXml object is a child of the jmri.jmrix.configurexml.AbstractSerialConnectionConfigXml class, which is in turn a child of the jmri.jmrix.configurexml.AbstractConnectionConfigXml class.
- After it's constructed, the ConnectionConfigManager calls `load(..)` on the ConnectionConfigXml object. This is implemented in jmri.jmrix.configurexml.AbstractSerialConnectionConfigXml which does:
  - Invoke `getInstance()` which initializes an `adapter` member implementing `SerialPortAdapter`. In this case, `getInstance()` is implemented in `jmri.jmrix.loconet.locobufferusb.configurexml.ConnectionConfigXml` and assigns a `jmri.jmrix.loconet.locobufferusb.LocoBufferUsbAdapter` to the "adapter" member of "ConnectionConfigXml" That's used later on to configure the port, etc.
  - Some load of serial port information, e.g. port name and speed values, followed by calling `loadCommon(shared, perNode, adapter)` from the base class, which brings in common information:
    - Values for four generic options: "option1" through "option4". The port adapters then convert this to specific, connection-specific options as needed later on
    - Calls `loadOptions(perNode.getChild("options"), perNode.getChild("options"), adapter)` to do any additional handling of info coded in an `<options>` element. Although overridden in some cases, the default for this is to invoke `adapter.setOptionState(name, value)` In this LocoNet case, that stores the command station name, see the element above.
    - Sets a "manufacturer" attribute/name in the TrafficController
    - Sets the system prefix and user name in the SystemConnectionMemo
    - Sets the adapter disabled or non-disabled
  - `register()`, which is implemented in `jmri.jmrix.loconet.locobufferusb.configurexml.ConnectionConfigXml` by invoking `this.register(new ConnectionConfig(adapter))`, which in turn is implemented in `jmri.jmrix.configurexml.AbstractConnectionConfigXml` as

                protected void register(ConnectionConfig c) {
                    c.register();
                }

    The `ConnectionConfig c` here is of type `jmri.jmrix.loconet.locobufferusb.ConnectionConfig` which extends `jmri.jmrix.AbstractSerialConnectionConfig` which extends `jmri.jmrix.AbstractConnectionConfig`. In `AbstractConnectionConfig`, finally, `register()` does:

                this.setInstance();
                InstanceManager.getDefault(jmri.ConfigureManager.class).registerPref(this);
                ConnectionConfigManager ccm = InstanceManager.getNullableDefault(ConnectionConfigManager.class);
                if (ccm != null) {
                    ccm.add(this);
                }

    That `this.setInstance()` call is implemented in `jmri.jmrix.loconet.locobufferusb.ConnectionConfig` to set the "adapter" member there to a new `LocoBufferUsbAdapter` object. Note that this "adapter" is from the ConnectionConfig (specifically AbstractConnectionConfig) object, not the ConnectionConfigXml object referred to above. In the sequence we're showing here, the `LocoBufferUsbAdapter` object had already been created by getInstance in `ConnectionConfigXml`, and passed to the `ConnectionConfig` object when it's created inside the `register()` sequence.

    At this point, we have a `jmri.jmrix.loconet.locobufferusb.ConnectionConfig` object registered for persistence, so it can be written out later.

  - Initialize the actual port using `adapter.openPort(portName, "JMRI")`. This uses code specific to the `adapter` member that was initialized in `getInstance()`, i.e. in this case LocoBuffer-USB code.
  - Finally, with the port open and available from the `adapter` object, initialize the operation of the system by calling `adapter.configure()` method. That Adapter configure() method does (through the general LocoBufferAdapter superclass) (this is given as a sample, ignore the details):

                setCommandStationType(getOptionState(option2Name));
                setTurnoutHandling(getOptionState(option3Name));
                // connect to a packetizing traffic controller
                LnPacketizer packets = new LnPacketizer();
                packets.connectPort(this);

                // create memo and load
                this.getSystemConnectionMemo().setLnTrafficController(packets);
                this.getSystemConnectionMemo().configureCommandStation(commandStationType,
                        mTurnoutNoRetry, mTurnoutExtraSpace);
                this.getSystemConnectionMemo().configureManagers();

                // start operation
                packets.startThreads();

    1. The first group does some internal housekeeping and creates the object(s) that run the connection
    2. The second group loads the previously-created SystemConnectionMemo with information about the connection.

       The `getSystemConnectionMemo` is in the common `LocoBufferAdapter` superclass. (There's some code in the inheritance chain that does some casting that should someday be cleaned up)

    3. The third group starts up operation

       At this point, the system is basically up and ready for operation.

    4. Finally, a jmri.jmrix.loconet.LocoNetSystemConnectionMemo object is created and registered with the InstanceManager.
- Later, jmri.jmrix.ActiveSystemsMenu and/or jmri.jmrix.SystemsMenu will create the main menu bar menus for the individual systems:
  - Ask the InstanceManager for all the ComponentFactory instances. These were created, in most cases, in the constructor of the SystemConnectionMemo, and live in the .swing subproject (e.g. loconet.swing.LnComponentFactory )
  - For each of those, ask it for the menu object (e.g. LocoNetMenu) and post that to the GUI.
  - In the process of creating the menu, the ComponentFactory connects each Action to itself so that the individual tools will be able to connect to the proper e.g. TrafficController, SlotMonitor, etc.
  - When an Action is fired later on, the invoked class(es) inquire of the LocoNetSystemConnectionMemo when they need a resource, instead of referring to an instance() method in the resource's class.

#### Lessons

Should this part move up?

- It's important that managers only be created once. More specifically, the managers and the SystemConnectionMemo should only be registered in the InstanceManager once. If they're registered more times than that, they appear as duplicates in various auto-constructed lists, menus and tab sets.
- Much work is done in the PortAdapter subclasses. From a common `jmrix.PortAdapter` interface, JMRI has two different forms for those: `jmrix.SerialPortAdapter` (Serial/USB connections) and `jmrix.NetworkPortAdapter` (network connections).

  Abstract base classes implement those as `jmrix.AbstractSerialPortController` (Serial/USB connections) and `jmrix.AbstractNetworkPortController` (network connections) (most, but not all, systems use one of those) with a common base of `jmrix.AbstractPortController`.

  These in turn are inherited into the system-specific classes, e.g `loconet.LnPortController` and `loconet.LnNetworkPortController` respectively (see UML diagrams on those linked Javadoc pages).

  Because Java doesn't allow multiple inheritance, the system-specific descendants of the two abstract base classes can't actually share a single common system-specific base class. This results in some code duplication in e.g. serial/USB connections vs the network connection classes in the system-specific classes.

  - The terminology switch from "PortAdapter" to "PortController" is confusing. In many cases, it's

                    Abstract*PortAdapter <- Sys*PortController <- Sys*PortAdapter

    as you work down the abstraction. This should eventually be fixed.
  - How does jmrix.AbstractStreamPortController fit into the `PortAdapter` class hierarchy? (there is no *StreamPortController as defined in its header; it extends AbstractPortController)

### More Complex Initialization: C/MRI

For a more complex example, consider C/MRI, which has more content in it's `<connection>`element:

      <connection userName="C/MRI" systemPrefix="C" manufacturer="C/MRI"
                        disabled="no" port="(none selected)" speed="9,600 baud"
                        class="jmri.jmrix.cmri.serial.sim.configurexml.ConnectionConfigXml">
        <options />
        <node name="0">
          <parameter name="nodetype">2</parameter>
          <parameter name="bitspercard">32</parameter>
          <parameter name="transmissiondelay">0</parameter>
          <parameter name="num2lsearchlights">0</parameter>
          <parameter name="pulsewidth">500</parameter>
          <parameter name="locsearchlightbits">000000000000000000000000000000000000000000000000</parameter>
          <parameter name="cardtypelocation">1122221112000000000000000000000000000000000000000000000000000000</parameter>
        </node>
        <node name="1">
          <parameter name="nodetype">1</parameter>
          <parameter name="bitspercard">24</parameter>
          <parameter name="transmissiondelay">0</parameter>
          <parameter name="num2lsearchlights">0</parameter>
          <parameter name="pulsewidth">500</parameter>
          <parameter name="locsearchlightbits">000000000000000000000000000000000000000000000000</parameter>
          <parameter name="cardtypelocation">2210000000000000000000000000000000000000000000000000000000000000</parameter>
        </node>
        <node name="2">
          <parameter name="nodetype">2</parameter>
          <parameter name="bitspercard">32</parameter>
          <parameter name="transmissiondelay">0</parameter>
          <parameter name="num2lsearchlights">0</parameter>
          <parameter name="pulsewidth">500</parameter>
          <parameter name="locsearchlightbits">000000000000000000000000000000000000000000000000</parameter>
          <parameter name="cardtypelocation">2212120000000000000000000000000000000000000000000000000000000000</parameter>
        </node>
        <node name="4">
          <parameter name="nodetype">1</parameter>
          <parameter name="bitspercard">24</parameter>
          <parameter name="transmissiondelay">0</parameter>
          <parameter name="num2lsearchlights">0</parameter>
          <parameter name="pulsewidth">500</parameter>
          <parameter name="locsearchlightbits">000000000000000000000000000000000000000000000000</parameter>
          <parameter name="cardtypelocation">2210000000000000000000000000000000000000000000000000000000000000</parameter>
        </node>
      </connection>

How this gets read in.

Implications for internal structure.

Proper internal structure

## Configuration Process

See jmrix.JmrixConfigPane Javadoc for links to configuration elements. (Is there another place that the configuration process and preferences support is described? If so, it should be linked from here.)

Any particular system connection is included in the preferences by being listed in the `java/src/META-INF/services/jmri.jmrix.ConnectionTypeList` list. This file is normally generated from the `@ServiceProvider(service = ConnectionTypeList)` class-level annotations.

See jmrix.JmrixConfigPane Javadoc for links to configuration elements. (Is there another place that the configuration process and preferences support is described? If so, it should be linked from here.)

Any particular system connection is included in the preferences by being listed in the `target/classes/META-INF/services/jmri.jmrix.ConnectionTypeList` list. This file is normally generated from the `@ServiceProvider(service = ConnectionTypeList)` class-level annotations.

    # Providers of System Connections type lists in Preferences
    # Order is Insignificant
    jmri.jmrix.internal.InternalConnectionTypeList
    jmri.jmrix.lenz.LenzConnectionTypeList
    ...
    jmri.jmrix.loconet.LnConnectionTypeList
    ...

This provides the contents for the 1st-level selection in the top JComboBox, e.g. in this case "Digitrax". This (generally) corresponds to selecting a system package within the JMRI package that might contain multiple variants of a specific connection. Within `JmrixConfigPane` this is called the "manufacturer" selection.

The contents of the `jmri.jmrix.loconet.LnConnectionTypeList`, an instance of `jmri.jmrix .ConnectionTypeList` then provides the contents for the second-level JComboBox of specific connection types, each corresponding (generally) to a specific `ConnectionConfig` implementation that can configure a specific connection type. Within `JmrixConfigPane` this is called the "mode" selection.

### Creating from scratch

Note this starts off by creating a ConnectionConfig, which creates a PortAdapter, similar to the read-from-XML version. But we don't want a running connection: We want one that we can work with to set/store configuration information. So, although we "register()", we do not "configure()".

Filling the `details` JPanel is done within the `ConnectionConfig` object via a call to `loadDetails()`. In many cases, including this LocoNet example, that's referred up to a base class:

- `AbstractSerialConnectionConfig` handles connections through serial links that need specification of serial port name, baud rate, etc.
- `AbstractNetworkConnectionConfig` handles connections through network (TCP) connections that need specification of network address, port, etc.
- `AbstractStreamConnectionConfig` handles configuration of connections based on streams.
- `AbstractSimulatorConnectionConfig` handles configuration of simulated connections.

### Storing

### Changing Options

The Swing panel that shows the main options (e.g. option1 through option4) sets changes to those values directly into the ConnectionConfig/PortAdapter without asking them to act further via e.g. configure().

### Updating a Connection Mode

Changing the mode JComboBox in `JmrixConfigPane` first clears the existing contents of the `details` JPanel with `removeAll()`, then calls the `JmrixConfigPane.selection()` method to refill it.

### Deleting a Connection

## Misc

This section is a grab-bag of other things you might want to know about the system connection structure.

- The `jmri.swing.ConnectionLabel` class is a Swing JLabel that listens to a single connection and displays its status. We use those on the main splash screen, but they can also be used in other places.