2.3 Describing extension modules

#
2.3 Describing extension modules

Just as writing Python extension modules is a bit more complicated than writing pure Python modules, describing them to the Distutils is a bit more complicated. Unlike pure modules, it's not enough just to list modules or packages and expect the Distutils to go out and find the right files; you have to specify the extension name, source file(s), and any compile/link requirements (include directories, libraries to link with, etc.).

All of this is done through another keyword argument to `setup()`, the ext_modules option. ext_modules is just a list of `Extension` instances, each of which describes a single extension module. Suppose your distribution includes a single extension, called `foo` and implemented by foo.c. If no additional instructions to the compiler/linker are needed, describing this extension is quite simple:

    Extension('foo', ['foo.c'])

The `Extension` class can be imported from `distutils.core` along with `setup()`. Thus, the setup script for a module distribution that contains only this one extension and nothing else might be:

    from distutils.core import setup, Extension
    setup(name='foo',
          version='1.0',
          ext_modules=[Extension('foo', ['foo.c'])],
          )

The `Extension` class (actually, the underlying extension-building machinery implemented by the `build_ext` command) supports a great deal of flexibility in describing Python extensions, which is explained in the following sections.

## 2.3.1 Extension names and packages

The first argument to the `Extension` constructor is always the name of the extension, including any package names. For example,

    Extension('foo', ['src/foo1.c', 'src/foo2.c'])

describes an extension that lives in the root package, while

    Extension('pkg.foo', ['src/foo1.c', 'src/foo2.c'])

describes the same extension in the `pkg` package. The source files and resulting object code are identical in both cases; the only difference is where in the filesystem (and therefore where in Python's namespace hierarchy) the resulting extension lives.

If you have a number of extensions all in the same package (or all under the same base package), use the ext_package keyword argument to `setup()`. For example,

    setup(...
          ext_package='pkg',
          ext_modules=[Extension('foo', ['foo.c']),
                       Extension('subpkg.bar', ['bar.c'])],
         )

will compile foo.c to the extension `pkg.foo`, and bar.c to `pkg.subpkg.bar`.

## 2.3.2 Extension source files

The second argument to the `Extension` constructor is a list of source files. Since the Distutils currently only support C, C++, and Objective-C extensions, these are normally C/C++/Objective-C source files. (Be sure to use appropriate extensions to distinguish C++ source files: .cc and .cpp seem to be recognized by both Unix and Windows compilers.)

However, you can also include SWIG interface (.i) files in the list; the `build_ext` command knows how to deal with SWIG extensions: it will run SWIG on the interface file and compile the resulting C/C++ file into your extension.

SWIG support is rough around the edges and largely untested; especially SWIG support for C++ extensions! Explain in more detail here when the interface firms up.

On some platforms, you can include non-source files that are processed by the compiler and included in your extension. Currently, this just means Windows message text (.mc) files and resource definition (.rc) files for Visual C++. These will be compiled to binary resource (.res) files and linked into the executable.

## 2.3.3 Preprocessor options

Three optional arguments to `Extension` will help if you need to specify include directories to search or preprocessor macros to define/undefine: `include_dirs`, `define_macros`, and `undef_macros`.

For example, if your extension requires header files in the include directory under your distribution root, use the `include_dirs` option:

    Extension('foo', ['foo.c'], include_dirs=['include'])

You can specify absolute directories there; if you know that your extension will only be built on Unix systems with X11R6 installed to /usr, you can get away with

    Extension('foo', ['foo.c'], include_dirs=['/usr/include/X11'])

You should avoid this sort of non-portable usage if you plan to distribute your code: it's probably better to write C code like

    #include <X11/Xlib.h>

If you need to include header files from some other Python extension, you can take advantage of the fact that header files are installed in a consistent way by the Distutils `install_header` command. For example, the Numerical Python header files are installed (on a standard Unix installation) to /usr/local/include/python1.5/Numerical. (The exact location will differ according to your platform and Python installation.) Since the Python include directory--/usr/local/include/python1.5 in this case--is always included in the search path when building Python extensions, the best approach is to write C code like

    #include <Numerical/arrayobject.h>

If you must put the Numerical include directory right into your header search path, though, you can find that directory using the Distutils `distutils.sysconfig` module:

    from distutils.sysconfig import get_python_inc
    incdir = os.path.join(get_python_inc(plat_specific=1), 'Numerical')
    setup(...,
          Extension(..., include_dirs=[incdir]),
          )

Even though this is quite portable--it will work on any Python installation, regardless of platform--it's probably easier to just write your C code in the sensible way.

You can define and undefine pre-processor macros with the `define_macros` and `undef_macros` options. `define_macros` takes a list of `(name, value)` tuples, where `name` is the name of the macro to define (a string) and `value` is its value: either a string or `None`. (Defining a macro `FOO` to `None` is the equivalent of a bare `#define FOO` in your C source: with most compilers, this sets `FOO` to the string `1`.) `undef_macros` is just a list of macros to undefine.

For example:

    Extension(...,
              define_macros=[('NDEBUG', '1'),
                             ('HAVE_STRFTIME', None)],
              undef_macros=['HAVE_FOO', 'HAVE_BAR'])

is the equivalent of having this at the top of every C source file:

    #define NDEBUG 1
    #define HAVE_STRFTIME
    #undef HAVE_FOO
    #undef HAVE_BAR

## 2.3.4 Library options

You can also specify the libraries to link against when building your extension, and the directories to search for those libraries. The `libraries` option is a list of libraries to link against, `library_dirs` is a list of directories to search for libraries at link-time, and `runtime_library_dirs` is a list of directories to search for shared (dynamically loaded) libraries at run-time.

For example, if you need to link against libraries known to be in the standard library search path on target systems

    Extension(...,
              libraries=['gdbm', 'readline'])

If you need to link with libraries in a non-standard location, you'll have to include the location in `library_dirs`:

    Extension(...,
              library_dirs=['/usr/X11R6/lib'],
              libraries=['X11', 'Xt'])

(Again, this sort of non-portable construct should be avoided if you intend to distribute your code.)

Should mention clib libraries here or somewhere else!

## 2.3.5 Other options

There are still some other options which can be used to handle special cases.

The extra_objects option is a list of object files to be passed to the linker. These files must not have extensions, as the default extension for the compiler is used.

extra_compile_args and extra_link_args can be used to specify additional command line options for the respective compiler and linker command lines.

export_symbols is only useful on Windows. It can contain a list of symbols (functions or variables) to be exported. This option is not needed when building compiled extensions: Distutils will automatically add `initmodule` to the list of exported symbols.