# E-MailRelay Developer Guide

## Principles

The main principles in the design of E-MailRelay can be summarised as:

- Functionality without imposing policy
- Minimal third-party dependencies
- Windows/Unix portability without #ifdefs
- Event-driven, non-blocking, single-threaded networking code
- Multi-threading optional

## Portability

The E-MailRelay code is written in Modern C++, superficially using C++17 but remaining compatible with C++11.

The header files gdef.h in src/glib is used to fix up some compiler portability issues such as missing standard types, non-standard system headers etc. Conditional compilation directives (#ifdef etc.) are largely confined this file in order to improve readability.

Windows/Unix portability is generally addressed by providing a common class declaration with two implementations. The implementations are put into separate source files with a _unix or _win32 suffix, and if necessary a 'pimple' (or 'Bridge') pattern is used to keep the o/s-specific details out of the header. If only small parts of the implementation are o/s-specific then there can be three source files per header. For example, gsocket.cpp, gsocket_win32.cpp and gsocket_unix.cpp in the src/gnet directory.

Underscores in source file names are used exclusively to indicate build-time alternatives.

## Event model

The E-MailRelay server uses non-blocking socket i/o, with a select() or epoll() event loop. This event model means that the server can handle multiple network connections simultaneously from a single thread, and even if multi-threading is disabled at build-time the only blocking occurs when external programs are executed (see --filter and --address-verifier).

The advantages of a non-blocking event model are discussed in the well-known C10K Problem document.

This event model can make the code more complicated than the equivalent multi-threaded approach since (for example) it is not possible to wait for a complete line of input to be received from a remote SMTP client because there might be other connections that need servicing half way through.

At higher levels the C++ slot/signal design pattern is used to propagate events between objects (not to be confused with operating system signals). The slot/signal implementation has been simplified compared to Qt or boost by not supporting signal multicasting, so each signal connects to no more than one slot. The implementation is now a thin wrapper over std::function.

The synchronous slot/signal pattern needs some care when when the signalling object gets destructed as a side-effect of raising a signal, and that situation can be non-obvious precisely because of the slot/signal code decoupling. In most cases signals are emitted at the end of a function and the stack unwinds back to the event loop immediately afterwards, but in other situations, particularly when emitting more than one signal, defensive measures are required (see glib/gcall.h).

## Module structure

The main C++ libraries in the E-MailRelay code base are as follows:

### glib

  Low-level classes for file-system abstraction, date and time representation, string utility functions, logging, command line parsing etc.

### gssl

  A thin layer over the third-party TLS libraries.

### gnet

  Network and event-loop classes.

### gauth

  Implements various authentication mechanisms.

### gsmtp

  SMTP protocol classes.

### gpop

  POP3 protocol classes.

### gstore

  Message store classes.

### gfilters

  Built-in filters.

### gverifiers

  Built-in address verifiers.

All of these libraries are portable between Unix-like systems and Windows.

Under Windows there is an additional library under src/win32 for the user interface implemented using the Microsoft Win32 API.

## SMTP class structure

The message-store functionality uses three abstract interfaces: MessageStore, NewMessage and StoredMessage. The NewMessage interface is used to create messages within the store, and the StoredMessage interface is used for reading and extracting messages from the store. The concrete implementation classes based on these interfaces are respectively FileStore, NewFile and StoredFile.

Protocol classes such as GSmtp::ServerProtocol receive network and timer events from their container and use an abstract Sender interface to send network data. This means that the protocols can be independent of the network and event loop framework.

The interaction between the SMTP server protocol class and the message store is mediated by the ProtocolMessage interface. Two main implementations of this interface are available: one for normal spooling (ProtocolMessageStore), and another for immediate forwarding (ProtocolMessageForward). The Decorator pattern is used whereby the forwarding class uses an instance of the storage class to do the message storing and filtering, while adding in an instance of the GSmtp::Client class to do the forwarding.

Message filtering (--filter) is implemented via an abstract GSmtp::Filter interface. Concrete implementations in the GFilters namespace are provided for doing nothing, running an external executable program, talking to an external network server, etc.

Address verifiers (--address-verifier) are implemented via an abstract GSmtp::Verifier interface, with concrete implementations in the GVerifiers namespace.

The protocol, processor and message-store interfaces are brought together by the high-level GSmtp::Server and GSmtp::Client classes. Dependency injection is used to provide them with concrete instances of the MessageStore, Filter and Verifier interfaces.

## Event handling and exceptions

The use of non-blocking i/o in the network library means that most processing operates within the context of an i/o event or timeout callback, so the top level of the call stack is nearly always the event loop code. This can make catching C++ exceptions a bit awkward compared to a multi-threaded approach because it is not possible to put a single catch block around a particular high-level feature.

The event loop delivers asynchronous socket events to the EventHandler interface, timer events to the TimerBase interface, and 'future' events to the FutureEventCallback interface. If any of the these event handlers throws an exception then the event loop catches it and delivers it back to an exception handler through the onException() method of an associated ExceptionHandler interface.

ExceptionHandler interface pointers are passed around in EventState structures. All event-handling classes generally accept an EventState in their constructor and they pass a copy to all base classes and contained sub-objects. The default ExceptionHandler just rethrows the current exception, which typically propagates back to main() and terminates the program.

However, sometimes there are objects that need to be more resilient to exceptions. In particular, a network server should not terminate just because one of its connections fails unexpectedly and a network client should not terminate just because the peer disconnects. In these cases the ExceptionHandler can be set up to be the owning parent object, which can can choose to simply delete the child object without rethrowing and killing the whole program. The GNet::Server and GNet::ClientPtr classes do this.

Event sources in the event loop are typically held as a file descriptor and a windows event handle, together known as a Descriptor. Event loop implementations typically watch a set of Descriptors for events and call the relevant EventHandler/ExceptionHandler code via the EventEmitter class.

EventState objects also contain a pointer to an EventLogging interface. This interface provides a string that describes some key attribute of the event handling object. EventLogging objects are arranged in a linked list that runs through the assemblage of event handling objects. Before delivering an event the EventEmitter combines the strings returned by this linked list and applies the result to the G::LogOutput singleton so that everything logged by the event handling code will have that prefix.

## Multi-threading

Multi-threading is used only to make DNS lookup and external program execution asynchronous. A std::thread worker thread is used in a future/promise pattern to wrap up the getaddrinfo() and waitpid() system calls. The shared state comprises only the parameters and return results from these system calls, and synchronisation back to the main thread uses the main event loop (see GNet::FutureEvent).

Multi-threading is also used in the Windows event loop once the number of handles goes above the WaitForMultipleObjects() limit.

## E-MailRelay GUI

The optional GUI program emailrelay-gui uses the Qt toolkit for its user interface components. The GUI can run as an installer or as a configuration helper, depending on whether it can find an installation payload. Refer to the comments in src/gui/guimain.cpp for more details.

The user interface runs as a stack of dialog-box pages with forward and back buttons at the bottom. Once the stack has been completed by the user then each page is asked to dump out its state as a set of key-value pairs (see src/gui/pages.cpp). These key-value pairs are processed by an installer class into a list of action objects (in the Command design pattern) and then the action objects are run in turn. In order to display the progress of the installation each action object is run within a timer callback so that the Qt framework gets a chance to update the display between each one.

During development the user interface pages and the installer can be tested separately since the interface between them is a simple text stream containing key-value pairs.

When run in configure mode the GUI normally ends up just editing the emailrelay.conf file (or emailrelay-start.bat on Windows) and/or the emailrelay.auth secrets file.

When run in install mode the GUI expects to unpack all the E-MailRelay files from the payload into target directories. The payload is a simple directory tree that lives alongside the GUI executable or inside the Mac application bundle, and it contains a configuration file to tell the installer where to copy its files.

When building the GUI program the library code shared with the main server executable is compiled separately so that different GUI-specific compiler options can be used. This is done as a 'unity build', using the pre-processor to concatenate the shared code into one source file and compiling that for the GUI. (This technique requires that private 'detail' namespaces are explicitly named rather than anonymous so that there cannot be any name clashes within the combined anonymous namespace.)

## Character encoding on Windows

E-MailRelay on Windows generally holds all its internal strings in UTF-8, independent of the current active code page or locale. This is relevant mostly to file system paths, but also to event viewer output, configuration file contents, command-lines assembled to run external programs, system account information, registry paths and environment variables.

The header file gnowide.h has inline functions that convert to and from UTF-8 before calling the wide Windows API functions. The actual convertion between UTF-8 and UTF-16 wide characters is done by the G::Convert class. As a temporary measure the G_ANSI pre-processor switch can be defined to go back to using ansi functions.

The G::Path class holds filesystem paths using UTF-8. Windows-specific source code, such as in gfile_win32.cpp, passes the UTF-8 strings to the nowide inline functions which in turn call wide runtime library functions like _wopen(). The exception is that the G::Path::iopath() method can be used to initialise std::fstreams directly, without using the nowide functions.

The G::Arg class can be used to capture the Windows command-line in its wide form and then convert to UTF-8. The main() and WinMain() functions use the G::Arg::windows() factory function to do this.

Configuration files are expected to use UTF-8 character encoding. The secrets file also notionally uses UTF-8, but Base64 or xtext encoding is used for the account details, so the encoding is less relevant there. The startup batch file (emailrelay-start.bat) necessarily uses the OEM code page and the E-MailRelay GUI now tries to ensure that the user's choice of install directory is compatible with this.

## Windows build

E-MailRelay can be built for Windows using the native Visual Studio MSVC compiler or using MinGW (mingw-w64) on Linux.

For active development use winbuild.bat to set up an environment that uses cmake and Visual Studio, or for one-off release builds use winbuildall.bat.

The winbuild.bat script expects to find mbedtls source code in a child or sibling directory and Qt libraries under c:\qt, but refer to winbuild.pm for the details. The build proceeds using cmake and cmake --build, resulting in statically-linked executables but with the GUI typically dynamically-linked.

The mbedtls code is built if necessary by running cmake and cmake --build in a mbedtls-x64 build sub-directory. The mbedtls headers are copied into the mbedtls build tree. The mbedtls configuration header (mbedtls_config.h) is optionally edited to enable TLS v1.3. If necessary delete the mbedtls-x64 build directory to trigger a rebuild.

A release assembly can be created by running winbuild-install.bat or perl winbuild.pl install. This makes use of the Qt windeployqt utility to assemble DLLs and it also generates the Qt .qm translation files.

For public release builds the E-MailRelay GUI must be statically linked. Start with a normal build with a dynamically-linked GUI and use winbuild.pl install to create the release assembly. Then use the qtbuild.pl perl script to build static Qt libraries from source in a location that winbuild.pl will find (or use winbuildall.bat). Rebuild so that the GUI is now statically linked and manually copy the statically-linked emailrelay-gui.exe binary into the release assembly, replacing emailrelay-setup.exe and emailrelay-gui.exe. Remove the now-redundant DLLs (in the both the root and payload directories) before zipping.

For MinGW cross-builds use ./configure.sh -w64 and make on a Linux box and copy the built executables. Any extra run-time files can be identified by running dumpbin /dependents in the normal way.

To target ancient versions of Windows start with a MinGW cross-build for 32-bit (./configure.sh -m -w32 --disable-gui). Then winbuild.pl install_winxp can be used to make a simplified distribution assembly, without a GUI.

## Windows packaging

On Windows E-MailRelay is packaged as a zip file containing the files assembled by winbuild.pl install with a statically-built GUI copied in manually (see above).

The setup program is the emailrelay GUI running in setup mode, with a payload directory containing the files to be installed. Refer to the comments in src/gui/guimain.cpp for more details.

## Unix build

E-MailRelay uses autoconf and automake, but the libexec/make2cmake script can be used to generate cmake files. The generated cmake files incorporate some of the settings from the configure script, so run configure or configure.sh before make2cmake. The configure script is normally part of the release but it can itself be generated by running the bootstrap script.

For a 'unity build' run configure (typically with compiler options passed via CXXFLAGS) and then make unity. Code-size optimisations such as -Os and -fwhole-program are particularly effective for a unity build. Refer to the comments in unity/Makefile.am for more information.

## Unix packaging

On Unix-like operating systems the native packaging system is normally used rather than the E-MailRelay GUI installer, so the configure script should be given the --disable-gui command-line option.

Top-level makefile targets dist, deb and rpm can be used to create a binary tarball, debian package, and RPM package respectively.

## Internationalisation

The GUI code has i18n support using the Qt framework, with the tr() function used throughout the GUI source code. The GUI main() function loads translations from the translations sub-directory (relative to the executable), although that can be overridden with the --qm command-line option.

The non-GUI code has some i18n support by using gettext() via the inline txt() and tx() functions defined in src/glib/ggettext.h. The configure script detects gettext support in the C run-time library, but without trying different compile and link options. See also po/Makefile.am.

On Windows the main server executable emailrelay.exe has a tabbed dialog-box as its user interface, but that does not have any support for i18n.

## Source control

The source code is stored in the SourceForge svn and/or git repository.

For example:

    $ svn co https://svn.code.sf.net/p/emailrelay/code emailrelay
    $ cd emailrelay/tags/V_2_6

or

    $ git clone https://git.code.sf.net/p/emailrelay/git emailrelay
    $ cd emailrelay
    $ git checkout V_2_6

Code that has been formally released will be tagged with a tag like V_2_6 and any post-release or back-ported fixes will be on a fixes branch like V_2_6_fixes.

## Compile-time features

Compile-time features can be selected with options passed to the configure script. These include the following:

- Configuration GUI (--enable-gui)
- Multi-threading (--enable-std-thread)
- TLS library (--with-openssl, --with-mbedtls)
- Debug-level logging (--enable-debug)
- Event loop using epoll (--enable-epoll)
- PAM support (--with-pam)

Use ./configure --help to see a complete list of options.