JMRI: Continuous Integration
Author: Bob Jacobsen
Keywords: JMRI technical code Continuous Integration jenkins

# JMRI Code: Continuous Integration

"Continuous Integration" is the process of rebuilding a system every time it changes, so that you rapidly learn of new problems and are motivated to remove existing ones.

The core of our system is a series of Jenkins jobs that routinely build the code, run checks, and creates downloadable installers. That means that we can and do put changes into our users hands very quickly. There's a CI status page that shows the combined status of that.

We divide these into three groups:

- Mandatory checks that must be OK before the change will be merged.
- Optional checks that provide additional information on the code. These don't have to be completely cleared before a change is merged, but their observations should be considered and their metrics should be improving.
- Independent checks that are run by our Jenkins instance on a periodic basis to look for deeper problems.

## Mandatory Checks

We use three CI engines to test every proposed change entered into our GitHub code repository before it gets merged.

- Travis
- GitHub Actions
- WIP check

### Travis CI

We use Travis CI to do integration/test builds on Linux for every pull into our main GitHub repositories.

Travis will build any branch that contains a .travis.yml file. The build history for the main JMRI repository is at https://travis-ci.org/JMRI/JMRI.

Travis normally runs several jobs in parallel:

- The "graphical" one, which just runs AllTest with schema checks and intermittent tests disabled to reduce runtime; this also does coverage checks. A screen buffer (not a real screen) is used for all the GUI tests.
- The "static" one, which runs various static code checks such as he checks for valid line-ends, the check for proper ASCII characters, SpotBugs, etc,
- The "headless" one, which runs HeadlessTest with schema tests enabled
- And one that separately and individual runs the tests that are marked as requiring separate runnning.

You can enable Travis CI for your fork (personal GitHub repository) as well. After merging JMRI/master into your own fork, allow Travis CI to monitor your fork, and it will begin testing after your next push to GitHub. Follow the instructions at Travis CI to set up your forks.

You can prevent a Travis CI from running against a specific commit by including "[ci skip]" (including the brackets, but not the quotes) anywhere in the commit message if so desired.

Travis is controlled by the .travis.yml file. Much of the work is done by the scripts/travis.sh script.

### GitHub Actions

We use GitHub Actions to do two things on every pull into our main GitHub repositories.

- Run a series of static checks: Run Spotbugs and the compiler warning checks, make sure Javadocs build cleanly, etc.
- Run the JUnit tests on a Windows machine.

These checks are defined by files in the .github/workflows/ directory.

These normally run both on PRs to the main JMRI/JMRI repository, but also on pushes and PRs to your own repository. If you want to turn that off,

1. On the page for your own repository, select "Settings"
2. From the left sidebar, select "Actions"
3. Pick on option for what can run. The bottom option turns everything off for you locally.

Note that if you turn this off, the "Actions" tab won't show in the GitHub web interface until you turn it back on.

### WIP

We use the WIP check to allow somebody to mark a PR as "Work in Progress" (WIP) by putting WIP in the title line. The PR will then not be merged until the WIP is removed.

## Optional Checks

We also run some advisory checkers. Although we don't require that they pass, we strongly recommend that people look at them and try to improve their metrics of test coverage, simplicity and understandability, etc.

- Coveralls
- CodeClimate

### Coveralls

The Coveralls check does a deep check of which parts of the code were run during the test steps, and which were left uncovered. This can be very useful information when writing tests to make sure your code is working properly. In general, you should try to increase the code coverage, even if just a bit, with every PR.

### Code Climate

The Code Climate points out places where the code appears to be complex in various ways. (It also provides coverage information, though Coveralls seems to do a better job of that.) We recommend you look at those results and make updates where they make sense, so that things are continuously improving, but not every suggestion it makes is worth it, or even appropriate. We don't require this to be clean before merging.

Code Climate is controlled by the .codeclimate.yml file.

## Independent Checks with Jenkins

JMRI uses the Jenkins continuous integration engine for integration and test builds. This includes keeping our web site up to date as changes are committed, building installer packages from development builds, and building final releases.

Our primary Jenkins engine is at https://builds.jmri.org/jenkins/. Because it's hosted outside the US, we force its Java locale to US English by setting the `LC_ALL` environment variable to `en_US` in the master Jenkins configuration settings.

Specific Jenkins results of interest:

- Builds page, showing status of the builds done after every series of commits. Successful builds are followed by builds installers via the Packages job.
- SpotBugs page, showing status of the more-extensive SpotBugs tests done periodically. There are also pages of the most recent changes and the most recent results.

  For more information on JMRI's use of SpotBugs, see the separate SpotBugs page.

### Jenkins Integration with NetBeans

If you are developing in the NetBeans 7.0 environment, it is possible to integrate feedback from the Jenkins Continuous Integration (CI) engine.

Within the NetBeans environment, reference is made to the Hudson CI engine - Jenkins is a "fork" of the Hudson code and supports the same API.

Integration into NetBeans is achieved by executing the following steps:

1. Open NetBeans
2. Switch to the 'Services' window (shortcut key 'Ctrl+5')
3. Right-click (Ctrl-click on Mac) the entry 'Hudson Builders' and choose 'Add Hudson Instance...'

4. In the resulting pop-up, complete the following:
   Name
     JMRI
   URL
     https://builds.jmri.org/jenkins/
   Auto refresh every X minutes
     60 (change from the default of 5 to avoid overloading the CI server)

From now on, the current status of the Jenkins builds can be reviewed in the 'Services' window by expanding the 'JMRI' entry under 'Hudson Builders'.