//----------------------------------------------------------------------------
//
// TSDuck - The MPEG Transport Stream Toolkit
// Copyright (c) 2005-2025, Thierry Lelegard
// BSD-2-Clause license, see LICENSE.txt file or https://tsduck.io/license
//
//----------------------------------------------------------------------------

[#automation]
=== Automation

TSDuck is a free open-source project which is maintained on spare time by very few developers
(only one at the time of this writing).
As a consequence, the maintenance of the project is driven by the lack of time and resource.
To face these constraints, automation is essential.

This is briefly explained in {home}download/docs/tsduck-project.pdf[this presentation].

For future maintainers or contributors, this page lists a few automation procedures which are used by the project.

[#auto_ci]
==== Continuous integration

Each time a commit or a pull request is pushed to GitHub, the workflow
`.github/workflows/continuous-integration.yml` is automatically run to verify
that the project can be built on most supported platforms and contains no regression.

* The workflow is run on Linux Ubuntu, macOS, and Windows (64 and 32 bits).
* The compilation is done using two revisions of the {cpp} language: {cpp}20 and {cpp}23.
* On Linux, the builds are repeated using two compilers, `gcc` and `clang`.
* In each configuration of platform, compiler and language level:
** TSDuck is fully built.
** The unitary tests are run.
** The full test-suite is downloaded and run.
** The sample programs are built using a typical TSDuck installation of the binaries which were just built (Linux and macOS only).
* The documentation is rebuilt: user's guide, developer's guide, programming reference using doxygen.
  Missing documentation for new features can be found in the log. This is done on Ubuntu only.

[#auto_nightly]
==== Nightly builds

Every night, if there were any modification in the TSDuck repo during the last day,
"nightly builds" are automatically produced and published on the {home}download/prerelease/[TSDuck web site].
No manual action is required, everything is automated.

The workflow `.github/workflows/nightly-build.yml` is automatically run every day at 00:40 UTC.

* On Linux Ubuntu and Windows (64 bits only), the TSDuck binary packages are built
  for the current state of the master branch in the repository.
* The produced binaries are installed on the CI/CD system.
* The full test-suite is downloaded and run.
* The complete set documentation (user's guide, developer's guide, programming reference)
  is built and packaged in an archive. If the installed version of doxygen is known
  to contain bugs in the generation of the TSDuck documentation (many were found),
  the source code for a known good version of doxygen is downloaded and rebuilt first.
  This is done to ensure that the documentation is always correct.
- The Linux Ubuntu binaries, the Windows binaries and the documentation package are
  published as "artefacts" of the workflow. They are publicly downloadable from
  {repo}actions/[GitHub].

At the end of the `nightly-build.yml` workflow, when all build jobs are completed,
the update job triggers a signal on the {home}[TSDuck web site].
A PHP script is automatically run on the web site to retrieve, download and publish the
latest {home}download/prerelease/[nighly binaries] and documentation.

[#auto_release]
==== Release creation

Creating an "official" TSDuck release is relatively easy.
Although the build time for all binaries can be long, everything is automated and
run in the background without requiring the maintainer's attention.

Since TSDuck is maintained with little resource and time, there is no product management cycle.
Thanks to the xref:auto_ci[continuous integration process],
any state of the repository can be used as a release candidate,
as long as there is no issue in the CI process and no major development or fix is in progress.

As a rule of thumb, a new release is produced every 3 to 6 months, when enough
new features or fixes are available, based on the
{home}download/changelog/[CHANGELOG file].

===== Building the various binaries

Binaries must be built for Windows (64 bits only) and a few major Linux distros
(Fedora, RedHat, Ubuntu, Debian, and 32-bit Raspian).
Additional binaries are now built for some Arm64 Linux distros.

The idea is to have a central system (Linux, macOS or Windows) from which the builds are orchestrated.
The script `pkg/build-remote.sh` can be used to build TSDuck binaries on various remote systems and
collect the resulting installers.

The remote systems can be physical systems or virtual machines running on the central system.
When a virtual machines is not active, it is automatically booted,
the binaries are built and collected, and then the virtual machines is shut down.
The currently supported hypervisors to managed virtual machines are VirtualBox, VMware and Parallels Desktop (macOS).
All systems shall be preconfigured so that the central user is authorized to connect on each of them using `ssh`
without password (use public key authentication, not passwordless accounts!)

The maintainer shall typically have a personal script, outside the repository,
which calls `build-remote.sh` on all systems.

Typical example of such a script, with one physical remote system (a Raspberry Pi)
and 5 virtual machines on the local host:

[source,shell]
----
$HOME/tsduck/pkg/build-remote.sh --host raspberry
$HOME/tsduck/pkg/build-remote.sh --host vmfedora --parallels Fedora
$HOME/tsduck/pkg/build-remote.sh --host vmredhat --parallels RedHat
$HOME/tsduck/pkg/build-remote.sh --host vmubuntu --parallels Ubuntu
$HOME/tsduck/pkg/build-remote.sh --host vmdebian --parallels Debian
$HOME/tsduck/pkg/build-remote.sh --host vmwindows --parallels Windows --windows --directory Documents/tsduck --timeout 20
----

So, building a release is as simple as running that script.
After "some time", all binaries and build log files are available in the `pkg/installers` subdirectory.

Be sure to check that all binaries are present.
Check the log files if some of them are missing.

===== Creating the GitHub release

Once the binary installers are collected, simply run this command to
create and publish the release on GitHub:

[source,shell]
----
$ pkg/github/release.py --create
----

The current state of the repo on GitHub is used as base for the release.
A tag is created with the version number.
The release is created with a explanatory description.
All binaries are published as assets of that release.

===== Creating the HomeBrew release

The GitHub release contains binaries for Linux and Windows.
On macOS, TSDuck is distributed through https://brew.sh/[HomeBrew].
The binaries for all macOS versions, Intel and Arm platforms, are built inside the HomeBrew CI/CD pipeline
when an updated formula is pushed in a pull request.

After the creation of the release on GitHub, create the pull request
for the new TSDuck version in HomeBrew using the following command:

[source,shell]
----
$ pkg/homebrew/brew-bump-formula-pr.sh -f
----

The option `-f` forces the creation of the pull request.
Without it, it works in "dry run" mode.
It can be safe to start with a dry run, as a test.

When the HomeBrew CI/CD pipeline has finished to process the pull request,
the new version of TSDuck is available for all flavors of macOS.

NOTE: Recently, TSDuck has been placed by the Homebrew maintainers in the
https://github.com/Homebrew/homebrew-core/blob/master/.github/autobump.txt["autobump" list of packages].
The original projects for the 2600+ packages in this list are regularly monitored by a Homebrew workflow.
Whenever one of these projects publishes a new version, the update of the formula is automatically done.
Next time a new version of TSDuck is published, it should be worth waiting a couple of days before
requesting an update using a pull request, to check if a similar pull request is automatically submitted
by the "autobump" workflow.

===== Updating the version number

Once a release is published, the minor version number of TSDuck `TS_VERSION_MINOR`
must be updated in the source file `src/libtscore/tsVersion.h`.

This is currently not automated and shall be manually updated before
the first commit following the publication of a new release.

[#auto_issues]
==== Cleanup of long-standing issues

The {repo}issues[issues area on GitHub] is used to report problems,
ask questions, and support any discussion about TSDuck.
When an issue is obviously completed, because a complete answer was provided or a fixed is pushed, the issue is closed.
Sometimes, a plausible response or fix is provided but some feedback is expected from the user to confirm this.
When a positive feedback is provided, the issue is closed.

However, some users never provide a feedback after their problem is solved.
In that case, the issue remains open forever.

To solve this, there is a label named "close pending".
When a plausible response, solution or fix is provided,
the maintainer of the project sets the "close pending" label on the issue.
It remains open.
However, if the issue is not updated in the next 150 days, it will be automatically closed.

This is achieved by the workflow `.github/workflows/cleanup-issues.yml`.
This workflow is scheduled every week on Sunday at 02:00 UTC.
It runs the Python script `pkg/github/close-pending.py`
which automatically closes all issues with label "close pending" and no update within the last 150 days.
