.. _bindings_generation:

===================
Bindings Generation
===================

Binding developers have two levels of access to cling via ``cppyy``:

* A simple command line interface.
* Automated generation of an end-user bindings package from a CMake-based
  project build.

Both are installed as part of the ``cppyy-backend`` component since they are
not needed by users of the bindings.

Command Line Interface
======================

rootcling
---------

This provides basic access to ``cling``::

    $ rootcling
    Usage: rootcling [-v][-v0-4] [-f] [out.cxx] [opts] file1.h[+][-][!] file2.h[+][-][!] ...[LinkDef.h]
    For more extensive help type: /usr/local/lib/python2.7/dist-packages/cppyy_backend/bin/rootcling -h

The basic mode of operation is to process the header files ('fileN.h')
according to certain `#pragmas in the LinkDef.h <https://root.cern.ch/root/html/guides/users-guide/AddingaClass.html#the-linkdef.h-file>`_
file in order to generate bindings accessible in Python under the 'cppyy.gbl'
namespace.

The output is

* A .cpp file (which, when compiled to a shared library)
* A .rootmap file
* A .pcm file

which are used at runtime by ``cling`` to expose the semantics expressed by the
header files to Python. Nominally, the compiled .cpp provides low-level Python
access to the library API defined by the header files, while ``cling`` uses the
other files to provide the rich features it supports. Thus, the shipping form
of the bindings contains:

* A shared library (which must be compiled from the .cpp)
* A .rootmap file
* A .pcm file

cling-config
------------

This is a small utility whose main purpose is to provide access to the
as-installed configuration of other components. For-example::

    $ cling-config --help
    Usage: cling-config [--cflags] [--cppflags] [--cmake]
    $ cling-config --cmake
    /usr/local/lib/python2.7/dist-packages/cppyy_backend/cmake

cppyy-generator
---------------

This is a clang-based utility program which takes a set of C++ header files
and generate a JSON output file describing the objects found in them. This
output is intended to support more convenient access to a set of
cppyy-supported bindings::

    $ cppyy-generator --help
    usage: cppyy-generator [-h] [-v] [--flags FLAGS] [--libclang LIBCLANG]
                           output sources [sources ...]
    ...

See the ``Cmake interface`` for details.

CMake interface
===============

The bindings generated by rootcling, are 'raw' in the sense that:

* The .cpp file be compiled. The required compilation steps are
  platform-dependent.
* The bindings are not packaged for distribution. Typically, users expect
  to have a pip-compatible package.
* The binding are in the 'cppyy.gbl' namespace. This is an inconvenience at
  best for users who might expect C++ code from KF5::Config to appear in
  Python via "import KF5.Config".
* The bindings are loaded lazily, which limits the discoverability of the
  content of the bindings.
* ``cppyy`` supports customization of the bindings via 'Pythonization' but
  there is no automated way to load them.

These issues are addressed by the CMake support. This is a blend of Python
packaging and CMake where CMake provides:

* Platform-independent scripting of the creation of a Python 'wheel' package
  for the bindings.
* An facility for CMake-based projects to automate the entire bindings
  generation process, including basic automated tests.

Python packaging of bindings
----------------------------

Modern Python packaging usage is based on the 'wheel'. This is places the onus
on the creation of binary artifacts in the package on the distributor. In this
case, this includes the platform-dependent steps necessary to compile the .cpp
file.

The generated package also takes advantage of the __init__.py load-time
mechanism to enhance the bindings:

* The bindings are rehosted in a "native" namespace so that C++ code from
  KF5::Config appears in Python via "import KF5.Config".
* (TBD) Load Pythonizations.

Both of these need/can use the output of the cppyy-generator (included in the
package) as well as other runtime support included in ``cppyy``.

CMake usage
-----------

The CMake usage is via two modules:

* FindLibClang.cmake provides some bootstrap support needed to locate clang.
  This is provided mostly as a temporary measure; hopefully upstream support
  will allow this to be eliminated in due course.
* FindCppyy.cmake provides the interface described further here.

Details of the usage of these modules is within the modules themselves, but
here is a summary of the usage. ``FindLibClang.cmake`` sets the following
variables:

::

    LibClang_FOUND             - True if libclang is found.
    LibClang_LIBRARY           - Clang library to link against.
    LibClang_VERSION           - Version number as a string (e.g. "3.9").
    LibClang_PYTHON_EXECUTABLE - Compatible python version.


``FindCppyy.cmake`` sets the following variables:

::

    Cppyy_FOUND - set to true if Cppyy is found
    Cppyy_DIR - the directory where Cppyy is installed
    Cppyy_EXECUTABLE - the path to the Cppyy executable
    Cppyy_INCLUDE_DIRS - Where to find the ROOT header files.
    Cppyy_VERSION - the version number of the Cppyy backend.

and also defines the following functions::

    cppyy_add_bindings - Generate a set of bindings from a set of header files.
    cppyy_find_pips - Return a list of available pip programs.

cppyy_add_bindings
^^^^^^^^^^^^^^^^^^

Generate a set of bindings from a set of header files. Somewhat like CMake's
add_library(), the output is a compiler target. In addition ancillary files
are also generated to allow a complete set of bindings to be compiled,
packaged and installed::

  cppyy_add_bindings(
      pkg
      pkg_version
      author
      author_email
      [URL url]
      [LICENSE license]
      [LANGUAGE_STANDARD std]
      [LINKDEFS linkdef...]
      [IMPORTS pcm...]
      [GENERATE_OPTIONS option...]
      [COMPILE_OPTIONS option...]
      [INCLUDE_DIRS dir...]
      [LINK_LIBRARIES library...]
      [H_DIRS H_DIRSectory]
      H_FILES h_file...)

The bindings are based on https://cppyy.readthedocs.io/en/latest/, and can be
used as per the documentation provided via the cppyy.gbl namespace. First add
the directory of the <pkg>.rootmap file to the LD_LIBRARY_PATH environment
variable, then "import cppyy; from cppyy.gbl import <some-C++-entity>".

Alternatively, use "import <pkg>". This convenience wrapper supports
"discovery" of the available C++ entities using, for example Python 3's command
line completion support.

The bindings are complete with a setup.py, supporting Wheel-based
packaging, and a test.py supporting pytest/nosetest sanity test of the bindings.

The bindings are generated/built/packaged using 3 environments:

- One compatible with the header files being bound. This is used to
  generate the generic C++ binding code (and some ancillary files) using
  a modified C++ compiler. The needed options must be compatible with the
  normal build environment of the header files.
- One to compile the generated, generic C++ binding code using a standard
  C++ compiler. The resulting library code is "universal" in that it is
  compatible with both Python2 and Python3.
- One to package the library and ancillary files into standard Python2/3
  wheel format. The packaging is done using native Python tooling.

+----------------------+---------------------------------------------------------------------------------------------+
|Arguments and options | Description                                                                                 |
+======================+=============================================================================================+
|pkg                   | The name of the package to generate. This can be either                                     |
|                      | of the form "simplename" (e.g. "Akonadi"), or of the                                        |
|                      | form "namespace.simplename" (e.g. "KF5.Akonadi").                                           |
+----------------------+---------------------------------------------------------------------------------------------+
|pkg_version           | The version of the package.                                                                 |
+----------------------+---------------------------------------------------------------------------------------------+
|author                | The name of the library author.                                                             |
+----------------------+---------------------------------------------------------------------------------------------+
|author_email          | The email address of the library author.                                                    |
+----------------------+---------------------------------------------------------------------------------------------+
|URL url               | The home page for the library. Default is                                                   |
|                      | "https://pypi.python.org/pypi/<pkg>".                                                       |
+----------------------+---------------------------------------------------------------------------------------------+
|LICENSE license       | The license, default is "LGPL 2.0".                                                         |
+----------------------+---------------------------------------------------------------------------------------------+
|LANGUAGE_STANDARD std | The version of C++ in use, "14" by default.                                                 |
+----------------------+---------------------------------------------------------------------------------------------+
|IMPORTS pcm           | Files which contain previously-generated bindings                                           |
|                      | which pkg depends on.                                                                       |
+----------------------+---------------------------------------------------------------------------------------------+
|GENERATE_OPTIONS optio| Options which are to be passed into the rootcling                                           |
|                      | command. For example, bindings which depend on Qt                                           |
|                      | may need "-D__PIC__;-Wno-macro-redefined" as per                                            |
|                      | https://sft.its.cern.ch/jira/browse/ROOT-8719.                                              |
+----------------------+---------------------------------------------------------------------------------------------+
|LINKDEFS def          | Files or lines which contain extra #pragma content                                          |
|                      | for the linkdef.h file used by rootcling. See                                               |
|                      | https://root.cern.ch/root/html/guides/users-guide/AddingaClass.html#the-linkdef.h-file.     |
|                      |                                                                                             |
|                      | In lines, literal semi-colons must be escaped: "\;".                                        |
+----------------------+---------------------------------------------------------------------------------------------+
|EXTRA_CODES code      | Files which contain extra code needed by the bindings.                                      |
|                      | Customization is by routines named "c13n_<something>";                                      |
|                      | each such routine is passed the module for <pkg>:                                           |
|                      |                                                                                             |
|                      | :: code-block python                                                                        |
|                      |                                                                                             |
|                      |     def c13n_doit(pkg_module):                                                              |
|                      |         print(pkg_module.__dict__)                                                          |
|                      |                                                                                             |
|                      | The files and individual routines within files are                                          |
|                      | processed in alphabetical order.                                                            |
+----------------------+---------------------------------------------------------------------------------------------+
|EXTRA_HEADERS hdr     | Files which contain extra headers needed by the bindings.                                   |
+----------------------+---------------------------------------------------------------------------------------------+
|EXTRA_PYTHONS py      | Files which contain extra Python code needed by the bindings.                               |
+----------------------+---------------------------------------------------------------------------------------------+
|COMPILE_OPTIONS option| Options which are to be passed into the compile/link                                        |
|                      | command.                                                                                    |
+----------------------+---------------------------------------------------------------------------------------------+
|INCLUDE_DIRS dir      | Include directories.                                                                        |
+----------------------+---------------------------------------------------------------------------------------------+
|LINK_LIBRARIES library| Libraries to link against.                                                                  |
+----------------------+---------------------------------------------------------------------------------------------+
|H_DIRS directory      | Base directories for H_FILES.                                                               |
+----------------------+---------------------------------------------------------------------------------------------+
|H_FILES h_file        | Header files for which to generate bindings in pkg.                                         |
|                      | Absolute filenames, or filenames relative to H_DIRS. All                                    |
|                      | definitions found directly in these files will contribute                                   |
|                      | to the bindings. (NOTE: This means that if "forwarding                                      |
|                      | headers" are present, the real "legacy" headers must be                                     |
|                      | specified as H_FILES).                                                                      |
|                      | All header files which contribute to a given C++ namespace                                  |
|                      | should be grouped into a single pkg to ensure a 1-to-1                                      |
|                      | mapping with the implementing Python class.                                                 |
+----------------------+---------------------------------------------------------------------------------------------+

Returns via PARENT_SCOPE variables::

  target              The CMake target used to build.
  setup_py            The setup.py script used to build or install pkg.

Examples::

  find_package(Qt5Core NO_MODULE)
  find_package(KF5KDcraw NO_MODULE)
  get_target_property(_H_DIRS KF5::KDcraw INTERFACE_INCLUDE_DIRECTORIES)
  get_target_property(_LINK_LIBRARIES KF5::KDcraw INTERFACE_LINK_LIBRARIES)
  set(_LINK_LIBRARIES KF5::KDcraw ${_LINK_LIBRARIES})
  include(${KF5KDcraw_DIR}/KF5KDcrawConfigVersion.cmake)

  cppyy_add_bindings(
      "KDCRAW" "${PACKAGE_VERSION}" "Shaheed" "srhaque@theiet.org"
      LANGUAGE_STANDARD "14"
      LINKDEFS "../linkdef_overrides.h"
      GENERATE_OPTIONS "-D__PIC__;-Wno-macro-redefined"
      INCLUDE_DIRS ${Qt5Core_INCLUDE_DIRS}
      LINK_LIBRARIES ${_LINK_LIBRARIES}
      H_DIRS ${_H_DIRS}
      H_FILES "dcrawinfocontainer.h;kdcraw.h;rawdecodingsettings.h;rawfiles.h")

There is a fuller example of embedding the use of cppyy_add_bindings for a
large set of bindings::

  https://cgit.kde.org/pykde5.git/plain/KF5/CMakeLists.txt?h=include_qt_binding

cppyy_find_pips
^^^^^^^^^^^^^^^

Return a list of available pip programs.
