2.1 The Basics

#
2.1 The Basics

The Python runtime sees all Python objects as variables of type `PyObject*`. A `PyObject` is not a very magnificent object - it just contains the refcount and a pointer to the object's ``type object''. This is where the action is; the type object determines which (C) functions get called when, for instance, an attribute gets looked up on an object or it is multiplied by another object. These C functions are called ``type methods'' to distinguish them from things like `[].append` (which we call ``object methods'').

So, if you want to define a new object type, you need to create a new type object.

This sort of thing can only be explained by example, so here's a minimal, but complete, module that defines a new type:

    #include <Python.h>

    typedef struct {
        PyObject_HEAD
        /* Type-specific fields go here. */
    } noddy_NoddyObject;

    static PyTypeObject noddy_NoddyType = {
        PyObject_HEAD_INIT(NULL)
        0,                         /*ob_size*/
        "noddy.Noddy",             /*tp_name*/
        sizeof(noddy_NoddyObject), /*tp_basicsize*/
        0,                         /*tp_itemsize*/
        0,                         /*tp_dealloc*/
        0,                         /*tp_print*/
        0,                         /*tp_getattr*/
        0,                         /*tp_setattr*/
        0,                         /*tp_compare*/
        0,                         /*tp_repr*/
        0,                         /*tp_as_number*/
        0,                         /*tp_as_sequence*/
        0,                         /*tp_as_mapping*/
        0,                         /*tp_hash */
        0,                         /*tp_call*/
        0,                         /*tp_str*/
        0,                         /*tp_getattro*/
        0,                         /*tp_setattro*/
        0,                         /*tp_as_buffer*/
        Py_TPFLAGS_DEFAULT,        /*tp_flags*/
        "Noddy objects",           /* tp_doc */
    };

    static PyMethodDef noddy_methods[] = {
        {NULL}  /* Sentinel */
    };

    #ifndef PyMODINIT_FUNC	/* declarations for DLL import/export */
    #define PyMODINIT_FUNC void
    #endif
    PyMODINIT_FUNC
    initnoddy(void)
    {
        PyObject* m;

        noddy_NoddyType.tp_new = PyType_GenericNew;
        if (PyType_Ready(&noddy_NoddyType) < 0)
            return;

        m = Py_InitModule3("noddy", noddy_methods,
                           "Example module that creates an extension type.");

        Py_INCREF(&noddy_NoddyType);
        PyModule_AddObject(m, "Noddy", (PyObject *)&noddy_NoddyType);
    }

Now that's quite a bit to take in at once, but hopefully bits will seem familiar from the last chapter.

The first bit that will be new is:

    typedef struct {
        PyObject_HEAD
    } noddy_NoddyObject;

This is what a Noddy object will contain--in this case, nothing more than every Python object contains, namely a refcount and a pointer to a type object. These are the fields the `PyObject_HEAD` macro brings in. The reason for the macro is to standardize the layout and to enable special debugging fields in debug builds. Note that there is no semicolon after the `PyObject_HEAD` macro; one is included in the macro definition. Be wary of adding one by accident; it's easy to do from habit, and your compiler might not complain, but someone else's probably will! (On Windows, MSVC is known to call this an error and refuse to compile the code.)

For contrast, let's take a look at the corresponding definition for standard Python integers:

    typedef struct {
        PyObject_HEAD
        long ob_ival;
    } PyIntObject;

Moving on, we come to the crunch -- the type object.

    static PyTypeObject noddy_NoddyType = {
        PyObject_HEAD_INIT(NULL)
        0,                         /*ob_size*/
        "noddy.Noddy",             /*tp_name*/
        sizeof(noddy_NoddyObject), /*tp_basicsize*/
        0,                         /*tp_itemsize*/
        0,                         /*tp_dealloc*/
        0,                         /*tp_print*/
        0,                         /*tp_getattr*/
        0,                         /*tp_setattr*/
        0,                         /*tp_compare*/
        0,                         /*tp_repr*/
        0,                         /*tp_as_number*/
        0,                         /*tp_as_sequence*/
        0,                         /*tp_as_mapping*/
        0,                         /*tp_hash */
        0,                         /*tp_call*/
        0,                         /*tp_str*/
        0,                         /*tp_getattro*/
        0,                         /*tp_setattro*/
        0,                         /*tp_as_buffer*/
        Py_TPFLAGS_DEFAULT,        /*tp_flags*/
        "Noddy objects",           /* tp_doc */
    };

Now if you go and look up the definition of `PyTypeObject` in object.h you'll see that it has many more fields that the definition above. The remaining fields will be filled with zeros by the C compiler, and it's common practice to not specify them explicitly unless you need them.

This is so important that we're going to pick the top of it apart still further:

        PyObject_HEAD_INIT(NULL)

This line is a bit of a wart; what we'd like to write is:

        PyObject_HEAD_INIT(&PyType_Type)

as the type of a type object is ``type'', but this isn't strictly conforming C and some compilers complain. Fortunately, this member will be filled in for us by `PyType_Ready()`.

        0,                          /* ob_size */

The `ob_size` field of the header is not used; its presence in the type structure is a historical artifact that is maintained for binary compatibility with extension modules compiled for older versions of Python. Always set this field to zero.

        "noddy.Noddy",              /* tp_name */

The name of our type. This will appear in the default textual representation of our objects and in some error messages, for example:

    >>> "" + noddy.new_noddy()
    Traceback (most recent call last):
      File "<stdin>", line 1, in ?
    TypeError: cannot add type "noddy.Noddy" to string

Note that the name is a dotted name that includes both the module name and the name of the type within the module. The module in this case is `noddy` and the type is `Noddy`, so we set the type name to `noddy.Noddy`.

        sizeof(noddy_NoddyObject),  /* tp_basicsize */

This is so that Python knows how much memory to allocate when you call `PyObject_New()`.

Note: If you want your type to be subclassable from Python, and your type has the same `tp_basicsize` as its base type, you may have problems with multiple inheritance. A Python subclass of your type will have to list your type first in its `__bases__`, or else it will not be able to call your type's `__new__` method without getting an error. You can avoid this problem by ensuring that your type has a larger value for `tp_basicsize` than its base type does. Most of the time, this will be true anyway, because either your base type will be `object`, or else you will be adding data members to your base type, and therefore increasing its size.

        0,                          /* tp_itemsize */

This has to do with variable length objects like lists and strings. Ignore this for now.

Skipping a number of type methods that we don't provide, we set the class flags to `Py_TPFLAGS_DEFAULT`.

        Py_TPFLAGS_DEFAULT,        /*tp_flags*/

All types should include this constant in their flags. It enables all of the members defined by the current version of Python.

We provide a doc string for the type in `tp_doc`.

        "Noddy objects",           /* tp_doc */

Now we get into the type methods, the things that make your objects different from the others. We aren't going to implement any of these in this version of the module. We'll expand this example later to have more interesting behavior.

For now, all we want to be able to do is to create new `Noddy` objects. To enable object creation, we have to provide a `tp_new` implementation. In this case, we can just use the default implementation provided by the API function `PyType_GenericNew()`. We'd like to just assign this to the `tp_new` slot, but we can't, for portability sake, On some platforms or compilers, we can't statically initialize a structure member with a function defined in another C module, so, instead, we'll assign the `tp_new` slot in the module initialization function just before calling `PyType_Ready()`:

        noddy_NoddyType.tp_new = PyType_GenericNew;
        if (PyType_Ready(&noddy_NoddyType) < 0)
            return;

All the other type methods are `NULL`, so we'll go over them later -- that's for a later section!

Everything else in the file should be familiar, except for some code in `initnoddy()`:

        if (PyType_Ready(&noddy_NoddyType) < 0)
            return;

This initializes the `Noddy` type, filing in a number of members, including `ob_type` that we initially set to `NULL`.

        PyModule_AddObject(m, "Noddy", (PyObject *)&noddy_NoddyType);

This adds the type to the module dictionary. This allows us to create `Noddy` instances by calling the `Noddy` class:

    >>> import noddy
    >>> mynoddy = noddy.Noddy()

That's it! All that remains is to build it; put the above code in a file called noddy.c and

    from distutils.core import setup, Extension
    setup(name="noddy", version="1.0",
          ext_modules=[Extension("noddy", ["noddy.c"])])

in a file called setup.py; then typing

    $ python setup.py build

at a shell should produce a file noddy.so in a subdirectory; move to that directory and fire up Python -- you should be able to `import noddy` and play around with Noddy objects.

That wasn't so hard, was it?

Of course, the current Noddy type is pretty uninteresting. It has no data and doesn't do anything. It can't even be subclassed.