JMRI Hardware Support - MQTT
Author: Bob Jacobsen
Keywords: JMRI MQTT

# Hardware Support: MQTT

MQTT is a "client-server publish-subscribe" protocol, meaning that JMRI connects to an MQTT server and subscribes to receive messages on certain topics. JMRI also publishes messages to the MQTT server and any device subscribed to that topic will receive a copy. This approach makes it relatively easy for the model railroader to use MQTT communication to develop any number of layout control and automation scenarios.

Devices both on and off the layout can use the MQTT protocol to connect to JMRI. As of release 4.21.1, JMRI implements MQTT-connected turnouts, sensors, and lights with additional object types expected in the future. For other objects and devices off the layout, you can use a provided script or write your own using JMRI's MQTT Adapter object and methods. You can use JMRI's MQTT capability to communicate with IOT devices as well as arduinos and other microcontrollers.

- Connecting MQTT Broker
- Layout Objects
- MQTT Topics
- Message Content
- Scripting
- Testing

## Connecting to the MQTT Message Broker

Establish a connection to an MQTT message broker via the Connections pane in the Preferences window.

- Select "MQTT" as the "System manufacturer"
- Select "MQTT Connection" as the "System Connection" (it is possible that "MQTT Simulator" may be added to this list in the future to allow testing without direct attachment to an MQTT broker).
- The "IP Address/Host Name" should be "localhost" if you are running an MQTT message broker on the same machine as JMRI, or the IP address or host name of an external server (see under Testing).
- The default "Connection Prefix" for MQTT is "M" (this can be changed when setting up the connection).
- The default "Connection Name" is "MQTT." Currently, JMRI can handle only one MQTT connection at a time. [This may be changed at some point necessitating the ability to change the Connection Name.]

You can establish an MQTT connection at the same time as other connection types.

[Go to top of page]

## Layout Objects

### Available Objects

Older versions of JMRI (4.12 and earlier) directly support Turnouts only. Sensors and Lights were added in version 4.21.1. Direct support for Reporters is expected to be added in version 4.22. Support for Signal Masts is planned but not yet scheduled.

MQTT messages can be sent and received and used with other object types, such as Memories, by writing custom scripts.

### Object Names

As with all objects within JMRI, directly supported layout objects (turnouts, sensors, and lights) using MQTT must have a "system name" and may have a "user name." If you are assigning the system name manually, the first letter must be the MQTT Connection Prefix. The next character must be an Object Type of "T", "S", or "L" for turnout, sensor, or light, respectively. The remainder of the name does not have to follow any particular model and can be any alphanumeric characters. For example, JMRI MQTT Turnouts can be specified with JMRI system names such as "MTabcd" or "MT123" or "MT.X$42".

If you create objects by "adding" them to their tables, select "MQTT" from the pulldown and the Connection Prefix and Object Type will automatically be inserted when the objects are created.

[Go to top of page]

## MQTT Topics

### Topics for Available Objects

Since JMRI 4.15.5 JMRI publishes and receives MQTT messages on specific "topics." An MQTT topic is the label or headline for a message that is broadcast or subscribed to. MQTT topics within JMRI are formed by concatenating:

- the base topic or "MQTT channel,"
- the object type-specific MQTT topic (may be different for publishing and subscribing), and
- the object's system name suffix.

The designated "MQTT channel" and object type-specific MQTT topics can be seen and changed by selecting "Additional Connection Settings" on the MQTT Connection page under Preferences.

Using the defaults in the JMRI Preferences Connection pane shown in the picture to the right, JMRI constructs the MQTT topic to send status of a Turnout with system name "MTabcd" by concatenating:

- the channel: "/trains/"
- the object-type specific "send" topic: "track/turnout/"
- the system name suffix: "abcd"

yielding the complete message topic: "/trains/track/turnout/abcd".

All default components can by changed on the MQTT connection tab (check the "Additional Connection Settings" box), and then entering the desired values. It is recommended that you change the default channel, typically to something based on your own railroad name. Current guidance from the MQTT community is not to have a leading slash; it is present in the JMRI default for historical reasons. [A good reason to change the default is that "/trains/" is the default for all users of JMRI. If you use a public MQTT message broker such as htttp://test.mosquitto.org test server (linked below), keeping that default channel will show you messages from every other JMRI instance anywhere in the world also using the default channel and connected to that broker -- and show everyone else your messages!]

You can change the object type-specific part of each topic using the other fields on the Preferences Connection page. There are separate fields for specifying what JMRI should publish (the "send" field) and what JMRI should subscribe to (recognize back from the layout, the "receive" field). The initial defaults contain two parts ("track/" and an object-specific part) and are the same for publish and subscribe but they need not be in this format. As noted below, identical send and receive topics are recommended for normal use.

If you want the object's system name suffix to be in the topic string at some place other than at the end, you can use a "wild card" of "{0}" within the string to put the object's system name suffix at that place. This allows you to set up topics like "track/turnout/{0}/state" so JMRI will generate "/trains/track/turnout/abc123/state" for turnout "MTabc123".

Be sure to save the JMRI Preferences if you have changed anything (hit the "Save" button on the Preferences list pane) and restart JMRI to put the changes into effect.

You can also change the object type-specific part of the topic strings as part of JMRI start-up using a script as described below.

### Topic Use

JMRI only uses some of the listed topics for its standard actions.

- Sensors: JMRI expects to receive sensor state on the "Sensor receive topic." It sends state on the "Sensor send topic" if a user changes sensor state by clicking on a panel or table.
- Turnouts: JMRI sends on the "Turnout send topic" if a user commands a turnout change by clicking on a panel or table. Additionally, if a Turnout has been created in either the Direct mode or Monitoring mode (see turnout documentation), JMRI expects to received turnout state on the "Turnout receive topic."
- Lights: JMRI sends on the "Light send topic" if a user commands a Light change by clicking on a panel or table. JMRI also acts on Light state messages sent from the layout on the "Light receive topic."
- Last Will: JMRI can specify its last will message when it connects to a broker. The last will message is a normal MQTT message with a topic. The broker stores the message until it detects that JMRI has disconnected ungracefully. In response to the ungraceful disconnect, the broker sends the last will message to all subscribed clients of the last will message topic. Find more information here about Last Will and Testament.

All other messages, even on defined topics, are ignored by the standard JMRI processing. To avoid confusion, it is recommended that send and receive topics are set the same except for advanced usage involving custom programmed layout microcontrollers. Any message topic, defined or custom, can be subscribed to and processed by a custom script.

### Custom Topics

It is possible to use the JMRI MQTT capability to send and receive messages at any time on a topic of your own choosing via a script (see below). These messages can further be used to trigger Logix, populate Memories and then displayed on panels, or whatever purposes you find useful.

[Go to top of page]

## Message Content

Message "payloads" for JMRI objects are typically single words that represent the state of the object. It is also possible to replace the standard message payloads by using a custom version of the message parser (including using JSON messages).

Note that some of the behavior listed in the following sub-sections is under review and future JMRI release (beyond version 4.21.3) may makes behaviors more consistent.

### Sensors

Since JMRI 4.21.1 The expected message content for sensors is "ACTIVE" and "INACTIVE". Any other message will generate a Warning in the JMRI System Console but will not change the Sensor state.

### Turnouts

The expected message content (send, also receive in Direct or Monitoring mode) for turnouts is "CLOSED" and "THROWN". Turnouts in Direct or Monitoring mode can also receive messages and have their Known State set to "INCONSISTENT" and "UNKNOWN". Receiving any other message will generate a Warning in the JMRI System Console and will set the turnout Known state to "UNKNOWN."

### Lights

Since JMRI 4.21.1 The expected message content for lights is "ON" and "OFF". Receiving any other messages will result in a Warning in the JMRI System Console. and cause the Light state to be set to "UNKNOWN."

[Go to top of page]

## Scripting

The JMRI MQTT capability is very general and can be used to send and receive arbitrary messages from any MQTT-aware device on or off the layout. In addition, scripts can be used to change standard message processing and default parameters.

All the scripts discussed here can be found in the JMRI distribution jython directory under "MQTT." More information about scripting with JMRI is provided under Scripting in the JMRI Help and web site. Custom scripts for MQTT will typically use the MQTT Adapter object and methods.

#### Changing the Default Message Parsing

Since JMRI 4.15.5

You can use a script to install a new MqttContentParser object to code and decode the content of messages. You might want to use JSON format, or need to adapt to specific messages from some already-existing device; you can create a custom parser to do any of that. See the SetMqttParser.py sample script. For a Java example, see the inner class implementation in MqttTurnout.

Note that you can call `setParser(...)` on the jmri.jmrix.mqtt.MqttTurnoutManager or on an individual jmri.jmrix.mqtt.MqttTurnout object. If you call it on the MqttTurnoutManager, all later created MqttTurnout objects will use the new parser; earlier-created ones will not be changed. This means you should execute a script to change the parser before loading any panel files if you want all MqttTurnouts to be modified.

#### Publishing Messages

The SendMqttMessage.py is an example of how to send any message on any topic (automatically prefixed with the JMRI MQTT channel) for processing by other devices.

#### Subscribing to (Receiving) Messages

You can use the ReceiveMqttMessage.py script to subscribe to any topic on the JMRI MQTT channel (both for JMRI objects and custom) to receive message for further processing. Standard MQTT wildcards (# and +) can be used (as of release 4.21.3).

#### Changing the MQTT Message Topic at JMRI Startup

Since JMRI 4.15.5

The sample script SetMqttPrefix.py shows how to set object type-specific topic components at JMRI startup. Note that any changes should be made at startup time before creating any Turnout objects, i.e. the script must be run before any panel files are loaded. Changing the prefix only affects Turnouts that are created after the change.

#### Changing MQTT Default Parameters

Since JMRI 4.21.3

Two parameters used in MQTT processing can be reset via scripts.

MQTT Quality of Service indicates to the MQTT message broker how to handle message delivery and receipt. Allowable values are 0, 1, and 2. JMRI uses QoS level 1 (broker will deliver message at least once) by default.

Retained Messages indicates to the MQTT message broker whether to retain the last message received for a certain topic. JMRI publishes messages with the retain option on. This allows any new subscriber to automatically receive the last message send to a topic (often the state of a JMRI object) when that new subscriber starts up.

Both of these parameters can be changed using the SetMqttOptions.py script.

[Go to top of page]

## Testing

If you don't already run an MQTT broker and want to test with this connection type, you can use a publicly-available (non-JMRI) test server. Enter "test.mosquitto.org" as the Host Name on the Preferences Connections page to connect to the host provided by mosquitto.org (for more information, type test.mosquitto.org into a browser.

You can also install the mosquitto tools on your own machine. You can then publish and subscribe to messages from JMRI.

        mosquitto_sub -h test.mosquitto.org -v -t '/trains/#'

to subscribe and print all JMRI messages as they are published. To publish messages for JMRI to pick up:

        mosquitto_pub -h test.mosquitto.org -t /trains/track/turnout/123 -r -m "CLOSED"
        mosquitto_pub -h test.mosquitto.org -t /trains/track/turnout/123 -r -m "THROWN"

These commands can be run on the same machine as JMRI, or on a separate machine.

You can also install a copy of the mosquitto broker by typing:

        mosquitto -v

and enter "localhost" as the Host Name on the JMRI Connection for MQTT. If you are using version 2.0 of mosquitto or later, you will need to provide a "configuration file":

        mosquitto -c "[location of your config file]/mosquitto.conf"

Include at least the following lines in mosquitto.conf:

        log_type all                 #Equivalent to setting -v (verbose mode)
        listener 1883                #To ensure listening on the appropriate port
        allow_anonymous true         #Allows JMRI to subscribe without an ID or password

See the mosquitto documentation for more information.

There are also MQTT tools available for iOS devices and Android devices that can be useful for monitoring, testing and operating your layout.

[Go to top of page]